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Abstract — This paper focuses on finite-dimensional upper and 
lower bounds on decodable thresholds of Zm and binary low- 
density parity-check (LDPC) codes, assuming belief propaga- 
tion decoding on memoryless channels. A concrete framework 
is presented, admitting systematic searches for new bounds. 
Two noise measures are considered: the Bhattacharyya noise 
parameter and the soft bit value for a maximum a posteriori 
probability (MAP) decoder on the uncoded channel. For Z„j 
LDPC codes, an iterative m-dimensional bound is derived for 
m-ary-input/symmetric-output channels, which gives a sufficient 
stability condition for Zm LDPC codes and is complemented by 
a matched necessary stability condition introduced herein. Appli- 
cations to coded modulation and to codes with non-equiprobable 
distributed codewords are also discussed. 

For binary codes, two new lower bounds are provided for 
symmetric channels, including a two-dimensional iterative bound 
and a one-dimensional non-iterative bound, the latter of which is 
the best known bound that is tight for binary symmetric channels 
(BSCs), and is a strict improvement over the bound derived 
by the channel degradation argument. By adopting the reverse 
channel perspective, upper and lower bounds on the decodable 
Bhattacharyya noise parameter are derived for non-symmetric 
channels, which coincides with the existing bound for symmetric 
channels. 

Index Terms — The Bhattacharyya noise parameter, the belief 
propagation algorithm, information combining, iterative decod- 
ing, LDPC codes, memoryless channels, non-symmetric channels, 
Zm alphabet. 

I. Introduction 

The belief propagation (BP)/sum-product algorithm [1] is 
one of the major components in modern capacity-approaching 
codes, including turbo codes [2], low-density parity-check 
(LDPC) codes [3], repeat accumulate (RA) codes [4], etc. 
The BP algorithm uses distributed local computation to ap- 
proximate the global maximum likelihood in an efficient way 
[5]. The density evolution (DE) method is a tool for explicitly 
computing the asymptotic behavior under iterative decoding 
with the assumption of independently distributed messages, 
which can be justified by the cycle-free convergence theorem 
in [6]. In each iteration, the DE method focuses on the 
density of the log likelihood ratio' (LLR), which is of infinite 
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'in the most general setting of the DE method, the quantity of interest 
during the iterations can be the density of any measure of the message passing 
decoder. Nevertheless, the density of the LLR is capable of capturing the entire 
behavior of the iterative decoder, since the density of the LLR is a sufficient 
statistic of the coiTesponding detection problem. 



dimension and is a sufficient statistic completely describing 
arbitrary binary-input memoryless channels. 

Even after the efficient implementation of density evolution 
by moving into the LLR domain, a one-dimensional iterative 
formula (or at most finite-dimensional formulae) to approxi- 
mate the density evolution is very appealing since it reduces 
significantly the computational complexity of code degree 
optimization [7]. Several approximation formulae have been 
proposed including Gaussian approximations [8], [9], binary 
erasure channel (EEC) approximations, reciprocal channel 
approximations [10], and the EXtrinsic Information Transfer 
(EXIT) chart analysis [11]. The finite dimensionality also helps 
in the analysis of the behavior of the message passing decoder 
[12], [13]. 

Contrary to the approximations, rigorous iterative upper 
and lower bounds generally sacrifice the threshold predictabil- 
ity for specific channel models in exchange for guaranteed 
universal performance for arbitrary channel models. Many 
results have been found for binary-input/symmetric-output 
(BI-SO) channels, including Burshtein et al. [13] on the soft 
bit value for the maximum a posteriori probability (MAP) 
decoder, Khandekar et al. [14] on the Bhattacharyya noise 
parameter, and Land et al. [15] and Sutskover et al. [16] 
on the mutual information. For binary-input/non-symmetric- 
output (BI-NSO) channels, a loose one-dimensional iterative 
upper bound on the Bhattacharyya noise parameter is provided 
in [17], which was used to derive the stability condition of BI- 
NSO channels and to upper bound the asymptotic convergence 
rate of the bit error probability. Bennatan et al. [18] used an 
iterative upper bound to derive the stability conditions for 
GF((7)-based LDPC codes when (jf is a power of a prime 
number 

This paper is organized as follows. The necessary definitions 
and background knowledge will be provided in Section HI] 
including the definitions of the symmetric channels, the noise 
measures of interest, and the LDPC code ensemble. SectionHIII 
will provide the framework for the iterative bounding problem 
and review some existing results. A Bhattacharyya-noise- 
parameter bound and a pair of stability conditions will be 
provided for Z„i LDPC codes in Section HVl For binary LDPC 
codes. Sections |V] and IVII are devoted to the iterative and 
non-iterative bounds respectively, the former of which include 
a one-dimensional bound for BI-NSO channels and a two- 
dimensional bound for BI-SO channels, while the latter of 
which provides the best (tightest) known bound for binary 
symmetric channels (BSCs). The existing bound based on the 
channel degradation argument [6] is also tight for BSCs, but is 
very loose for other channels, compared to which our bound 
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is a strict improvement and generates much tighter results for 
other channel models. Performance comparisons are provided 
in Section IvTII Section IVUII concludes the paper. 

II. Formulation 

In this paper, we consider only memoryless channels with 
discrete input alphabets. 

A. Symmetric Channels 

1 ) Definition: A BI-SO X i-^ Y channel is conventionally 
defined as a channel with binary^ input set X = {0, 1} and 
real output set Y = K, such that P{Y = y\X = 0) = P{Y = 
—y\X = 1) where X and Y denote the (random) channel input 
and output, respectively. In the literature of LDPC codes (eg., 
[6]), an equivalent commonly-used definition is that the BI- 
SO channel satisfies dP{m) = e"^dP{—m), where dP{m) is 
the density of the LLR messages, m :— log pTpw^jy, given 
X = 0. 

Let Zm :={0,1,---,TO— 1} denote the integer ring mod- 
ulo TO. A more general definition for TO-ary-input/symmetric- 
output (MI-SO) channels is given as follows. 

Definition 1 (MI-SO Channels): For any function T : Y i-^ 
Y, let 



denote the corresponding fc-times self-composition of T. An 
TO-ary-input channel Z„i i-^ Y is (circularly) symmetric if 
there exists a bijective transform T : Y i-^ Y such that 

"^"(y) = 2/,Vj/ e Y and 

VxGZ,„,F(d2/|0) = ^^(r-(dy)|x), 

where F(dy\x) is the conditional distribution of F G given 
X = X. When m = 2, this definition collapses to that of the 
conventional BI-SO channel. 

Note: There is no constraint on Y, the range of the channel 
output. For example, in phase-shift keying (PSK) or quadrature 
amplitude modulation (QAM) scenarios, Y = R^. 

This definition of "symmetric channels" coincides with the 
definition of the "matched signal set" in [19]. It is worth 
noting that belief propagation on LDPC codes is (circularly) 
symmetric, since the locally optimal BP decoder behaves 
identically under different transmitted codewords when the 
same circularly shifted error pattern is received. To be more 
explicit, assume that a non-zero codeword x = (xi, • • • , a;„) 
is transmitted and the received likelihood values are p(y) = 
(p(yi), • • • ,P(yn)). where each coordinate p(yj) = {P{Yi = 
y,\X, = 0),P(y, = y,\X, = l),...,P(y, = y,\X, = 
TO — 1)) is a vector containing the likelihood values of xi after 
receiving yi. The circular symmetry of the belief propagation 
decoder is characterized by the fact that the decoding behavior 
under the codeword and likelihood pair (x, p(y)) is identical 
to the case in which the all-zero codeword is transmitted 

^Another common setting is to consider X = {+1, —1}, wliicli reflects 
colierent binai'y phase stiift keying (BPSK) modulation. However, to be 
compatible with the algebra on which the parity check equations are defined, 
we assume X = {0, 1} instead of {+1, — !}■ 
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(a) An MI-NSO channel. 
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(b) A symmetrized MI-NSO channel. 
Fig. 1. Channel symmetrization. 



and the shifted UkeUhood p' 
is received, where 



^(y) = (P^^yi),--- ,P""(2/n)) 



P"'iy^) ={P{yr = y^\X^ = Xi),P{Y, = y,\X, = + 1), 

• • • , P(y, = y,\X, = TO - l),P{Y, = y,\X, = 0), 
••• ,P{Y^y^\X, = x^-l)). 

One immediate benefit of considering a symmetric channel 
LDPC codes is that all codewords have the same error prob- 
ability under the BP decoder and we can use the all-zero 
codeword as a representative, which facilitates the simulation 
of codes with finite length. Further discussion on the BP 
decoder for LDPC codes and on the representative all- 
zero codeword can be found in [20] and in [19], [6], [17]. 

One advantage of Definition^is that we can immediately 
prove the channel symmetrizing argument as follows. Consider 
an TO-ary-input/non-symmetric-output (MI-NSO) channel in 
Fig. |l(a)| and a concatenated new channel in Fig. |l(b)| sharing 
the same MI-NSO channel block. Since the receiver of the 
latter channel is able to use the received value of W to 
invert this concatenation, these two channels are equivalent 
from the detection point of view, which in turn implies 
that all reasonable noise measures of the two are identical, 
including but not limited to the channel capacities, the error 
probabilitiess and soft-bit values under the MAP decoder, and 
the Bhattacharyya noise parameters. The circular symmetry of 
this new equivalent channel X i-^ {W, Y) in Fig. |l(b)| can then 
be verified by letting the bijective transform T : (Z^, Y) i-^ 
(Z„j,Y) in Definition ^\ he T{w,y) = (w — l,y). From 
the above discussion. Fig. |l(b)| is an equivalent, symmetrized 
version of the original MI-NSO channel, and we can assume 
all channels are symmetric as long as the additional complexity 
of the channel symmetrizing'' is bearable. 

2) MSC Decomposition: One of the simplest MI-SO chan- 
nels is the m-ary symmetric channel (MSC), which is a Z„i ^ 
Z,„ channel and can be fully specified by a parameter vector 

^This channel symmetrizing technique is equivalent to considering the 
LDPC coset code ensemble [21]. 
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MI-SO Channel 




(a) By Conditional Distributions. 



MI-SO Channel 




X £ Z. 



(b) By Probabilistic Combinations. 



Fig. 2. Different representations for the m-ary-input/symmetric output 
channels. 



p = {po,Pi, ■ ■ ■ such that the conditional probability 

P{Y ~ X + i\X = x) = Pi, Vx, i £ TLra- From Definition 
it can be proved that any MI-SO channel can be uniquely 
expressed as a probabilistic combination of different MSCs, 
while p is observed by the receiver as side information. This 
decomposition is illustrated in Figs. |2(a)| and |2(b)| in which 
the probabilistic weight of different vectors p is denoted as 
dP{p), and a formal proof of this MSC decomposition is given 
in Appendix m When m = 2, an MSC collapses to a BSC 
and the channel-specifying vector p equals (1 —p,p), where 
p is the crossover probability. For simplicity, we sometimes 
use the scalar parameter p rather than the vector p to specify 
a BSC. 

Note: The probabilistic weight dP{p) does not depend on 
the a priori input distribution on X, but only depends on the 
channel model P(F|X). This observation will be used in the 
proof of the non-iterative bound in Section IVII 



can be found in [22]. With uniformly distribution on X, 
CB can be related to the cutoff rate hy Rq ^ 1 — 
log2(CB + 1). 
[The Soft Bit Value] 



SB ~2Ex.Y {p{X\Y)} 

=2 J2 P{X = x)E{pix\Y)\X 



, (2) 



xe{o,i} 

which was used in the bounds of [13]. 

Each of the above noise measures has the property that the 
condition CB = (or SB ~ 0), represents the noise-free 
channel, while CB = 1 (or SB = 1), implies the noisiest 
channel in which the output is independent of the input. It is 
worth noting that both CB and SB are well defined even 
for BI-NSO channels with non-uniform input distributions. 
Most of our theorems are derived based on the assumption 
of uniformly distributed X, and special notes will be given 
when non-uniform a priori distributions are considered. 

For BSCs with uniformly distributed X, CB = 2^p(l - p) 
and SB = 4p(l — p) where p is the crossover probability. By 
the BSC decomposition argument in Section Hi- A. 21 the value 
of CB or SB for any BI-SO channel is simply the probabilistic 
average of the corresponding values of the constituent BSCs, 
that is, for uniformly distributed X 

CB = j 2^p{l-p)dP{p) 
SB = j Ap{l-p)dP{p). 

The above formulae will be extensively used in our derivation 
of finite dimensional bounds. In the context of density evo- 
lution [6], CB and SB with uniformly distributed X can be 
expressed as 



CB = 
SB = 



e 2 



dP{m) 



m— — oo 



1 + e' 



-dP{m), 



(3) 



B. Noise Measures 

1) Binary Input Channels: For a binary channel X = 
{0, 1} 1-^ Y, we use p{x\y) to denote the a posteriori 
probability P{X = x\Y = y), and consider the following 
two noise measures: 

• [The Bhattacharyya Noise Parameter (the Chernoff bound 
value)] 

where x denotes the complement of the binary input x. 
A discussion of the CB parameter in turbo-like codes 



where m log p|yj^~°j is the passed LLR message and 
dP{m) dP{m\X = 0) is the density of m given X = 0. 
With the assumption of uniformly distributed X, the CB and 
SB values of some common channel models are given as 
follows (listed in order from the most BSC-like to the most 
BEC-like^): 

1) The BSC with crossover probability p: 

CB - 2^p{l-p)=CB{p) 
SB = 4p{l-p) = SB{p). 

''The order is obtained by plotting the {CB, SB) values of different channel 
models of the same capacity in a two-dimensional plane, similar to Fig. Q 
The channel models are then sorted according to their distances related to the 
points coiTesponding to a BSC and a BEC. 
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2) The binary-input Laplace channel (BiLC) with vari- 
ance 2A2, i.e. pL{y) = ^ cxp (^-x)- 



CB 



SB = 



1 + A 

A 
cxp I 



■cxp [ -- 

i) ^ 



cosh(i] 



2 exp I — -- I arctan f tanh ( — - 
V A/ V V2A 



3) The binary-input additive white Gaussian channel (Bi- 
AWGNC) with noise variance a^: 



CB 



SB 



exp 



V2^ 



2a\ 
: exp 



1 

'2^ 



exp 



cosh (^) 



dx. 



4) The binary-input Rayleigh fading channel with unit input 
energy and noise variance a^, i.e. the density function 
of the output amplitude is pyi(a) = 2aexp(— a^) and 
the additve noise distribution is J\f{0, cr^): 

CB ' 
SB 



1 



1 

2cr2 



V2^ 



exp 



a exp I 



-a 



/o ^-oo cosh I 

5) The EEC with erasure probability e: 

CB = SB^ €. 



7^) 



dxda. 



2) m-ary Input Channels: For m-ary-input channels, we 
define the pairwise Bhattacharyya noise parameter from x to 
x' as follows; 



CB{x -> x') 




P{x'\Y) 



P{x\Y) 



X 



(4) 



Considering any MI-SO channel with uniformly distributed 
input X, we immediately have 



[Symmetry:] CB{x x') ^ CB{x' x) 
[Stationarity:] CB{x ^ x') = CB{Q x' ~ x). 



(5) 



By stationarity, we can then use CB := {CB(Q ^')}x'ez 
as the representing vector for all CB(x x'). Also assuming 
the uniform distribution on X, the cutoff rate i?o and CB can 
be related as follows [23]: 



i?o = log2 m - log2 ( ^^(0 ) 



(6) 



Example: 

• For an MSC with parameter p and uniformly distributed 
X, we have 



When m = 2, the representing vector becomes CB = 
(l,CB(p)), where CB{p) = 2^p{l-p) is the tradi- 
tional Bhattacharyya noise parameter for BSCs. 



C. Error Probability vs. CB vs. SB 

Let = P {^X 7^ XMApiY)^ denote the error probability 
of the MAP decoder The relationship between p^ and the 
above noise measures CB (or CB) and SB are stated by the 
following lemmas. 

Lemma 1: For general BI-NSO channels and arbitrary input 
distributions, we have 



and 



2pe< CB < 2v/pe(l -Pe) 
2pe< SB <4pe(l-Pe) 

SB< CB < VSB. 



Lemma 2: For any MI-SO channel with uniform input 
distribution, we have 

2pe < CB{0^x). 

2;eZm\{0} 

If Pe < 1/2, then 



max CB(0 

EZ„\{0} 



V) < 2v/pe(l-Pe). 



Lemma ^guarantees that the three statements: pe — > 0, CB 
0, and SB are equivalent. Lemma |2] guarantees pe ^ 
is equivalent to the statement that Vx G Z„j\{0}, CB{0 
x) 0. Detailed proofs of Lemmato and |5] are provided in 
Appendix im 

D. The Equiprobable Graph Ensemble for LDPC Codes 

Throughout this paper, we consider only the equiprobable 
graph ensemble for LDPC codes [6], where each element 
corresponds to the Tanner graph of the parity check matrix. 
The construction of the equiprobable graph ensemble is as 
follows. Consider two finite sets of nodes: variable nodes and 
check nodes, in which each node is assigned a degree dcg(w) 

(or deg(c)) such that X;all u deg(v) ^ J^all cA'^Sic) ^ "e- 
Assign dcg(u) sockets to each variable node v and index all 
variable sockets from 1 to Ue- Assign deg(c) sockets to each 
check node c and index all check sockets from 1 to n^- Let 
7r(-) be an arbitrary permutation of the integers 1, • • • ,ne- 
Construct an edge connecting variable socket i and check 
socket j iff i = Ti{j). The above procedure results in a bipartite 
graph and the equiprobable graph ensemble is constructed by 
letting 7r(-) being drawn from a uniform random permutation 
ensemble. 

Based on this construction, we can define the edge-degree 
polynomials \{x) = X^fc-^fca;'^"^ and p{x) = Ylk Pk^''~^ ^ 
where Afc (or pk) is the percentage of the edges connecting to 
a variable (or check) node of degree k. This code ensemble can 
then be uniquely specified by the degree polynomials A and p 
and the codeword length n, and is denoted by C"(A,/9). We 
will sometimes use C(A, p) to denote the asymptotic ensemble 
in the limit of large n. Further discussion of the asymptotic 
ensemble can be found in [6]. 
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i=l 2 3 4 5 6 

(a) The corresponding Tanner graph. 




MI-SO 



MI-SO 



(b) The embedded support 
tree of depth 21, I = 1. 



Fig. 3. Supporting tree of a regular (2,3) code. 
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Fig. 4. LDPC codes with channels being decomposed as the probabilistic 
combinations of MSCs. 



III. The Support Tree Channel & Existing Results 
A. The Support Tree Channel 

Due to the inherent nature of message-exchanging during 
each iteration, the resuh of a BP decoder after / iterations 
depends only on the neighbors (of the target variable node) 
within a distance of 21. With the assumption that there is 
no cycle of length less than 21 in the corresponding Tanner 
graph, which holds for sufficiently large codeword length n 
in probability, the BP decoder on LDPC codes can be broken 
down into a tree structure of depth 21 as shown in [6] and 
demonstrated in Fig.|3l which considers the simplest case with 
target variable xi and I = 1. The arrows in Fig. |3(b)| represent 
the message flow in the decoding process. 

By the MSC decomposition argument, the tree structure can 
be viewed as in Fig. |4] which is a i-^ ([0, 1]™ x 
vector channel. The arrows are now pointing in the oppo- 
site direction since they now represent the data flow during 
transmission. Due to its optimality when applied to a cycle- 
free inference network, the BP decoder is in essence an 
efficient version of a MAP decoder on the tree structure. 
Therefore, it is more convenient to focus on the behavior 
of general MAP decoders on this tree-like vector channel, 
instead of considering the message passing behavior of the BP 
decoder. It is worth emphasizing that throughout this paper. 



only independently and identically distributed channels are 
considered, and all dPi, dP^, dPe and their corresponding 
MSCs are independent. 

Our target problem is to find bounds on the noise measures 
of the Z„i 1-^ ([0,1]™ X Z,„)'^ vector output channel, given 
constraints of finite-dimensional noise measures on the con- 
stituent channel distribution dPi{p), i = 1,5,6. To simplify 
the problem further, we consider the variable node and the 
check node channels respectively as in Figs. |5(a)| and |5(b )| in 
which the new constituent Xi t-^ (q, Y') channel in Fig. |5(a)| 
represents the entire Xi i-^ (ps, Y^) x (pg, Yq) vector channel 
in Fig. |5(b)| Once the noise measure of the Xi i-^ (psj^s) x 
(pe, Yq) check node channel is bounded given the constraints 
on Xr, 1-^ (P5,i^5) and X^ i-^ {pa^Yg), this newly obtained 
bound for Fig. |5(b)| can serve as a constraint on the constituent 
channel, Xi i-^ (q, Y'), of the variable node channel in 
Fig. |5(a)| When considering the behavior after infinitely many 
iterations, we can iteratively apply these check/variable node 
bounding techniques by switching the roles of "bounds on 
the vector output channel" and "constraints on the constituent 
channels" as in the aforementioned example. Given an initial 
constraint on the finite-dimensional noise measure of the 
constituent channels, whether the LDPC code is decodable can 
be determined by testing whether the noise measure converges 
to zero or is bounded away from zero as iterations proceed, 
which in turn gives us finite-dimensional lower/upper bounds 
on the decodable threshold. 

For variable/check nodes with degrees d > 3, if we take the 
marginal approach (focusing on one input constituent channel 
while leaving other constituent channels fixed), all the effects 
of the fixed inputs can be grouped into a single input message. 
Therefore, it is as if we take the marginal approach on a 
variable/check node with degree equal to three. The analysis 
of nodes of degree one or two is trivial. As a result, throughout 
this paper, only nodes of degree d — 3 will be discussed in 
detail, and the variable/check node channels of interest are 
illustrated in Figs. |5(c)| and |5(d)| with inputs/outputs relabelled 
for easier reference. 



B. Existing Results on Binary LDPC Codes 

For BI-SO channels, the best way to explain the existing 
results in [13], [14], [15], and [16] is using the idea of 
"transfer functions" and the convexity/concavity analysis. In 
this subsection, we will consider only the noise measure CB 
for example, which will lead to the iterative upper bound 
in [14] and a new iterative lower bound. Similar arguments 
can be used to derive the results in [13] or in [15], [16], if we 
substitute either SB or the conditional entropy for the noise 
measure CB. 

1) Check Nodes: For a check node as in Fig. |5(d)| the 
problem of finding an iterative upper/lower bound can be cast 
as an optimization problem as follows. 

max or min CBout= J CBchk{p,q)dP{p)dQ{q) (7) 

subject to CB,n,i = J CBi{p)dP{p) 
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(a) The variable node chan- 
nel of Fig. 



(b) The check node channel 

of Fig. m 



(c) A var. channel with re- 
labelled input/output. 



(d) A chk. channel with re- 
labelled input/output. 



Fig. 5. Separate consideration of vaiiable and check nodes. 



in. 2 



CB2{q)dQ{q), 



where 



CB,hkip,q) = 2^p{l ~p)+ q{l - q) ~ 4p(l - p)q{l - q) 

(8) 



CB^{p)=2^p{l-p) 
CB^iq) = 2v/g(l-<z). 

CBchk {p, q) denotes the value of CB for the tree-like 
check node channel if both the constituent channels are 
BSCs with parameters p and q respectively, and CBi{p) and 
CB2{q) denote the CB values of the constituent BSCs with 
parameters p and q. Using some simple algebra, it can be 
shown that for fixed p and q, the X ^ Yi x Y2 check 
channel is equivalent to a BSC with parameter tt = p{l 

9) + 

2^p{l-p) + q{l-q) 



p)q. Therefore CBchk{p,q) 



^2./^- it) 
q), from which 



4p{l-p)q{l 

follows. Furthermore, by omitting the input parameters p 
and q, we can rewrite CBchk {Pi q) in terms of CBi (p) and 
CB^iq) by 



CBchk — yC'Bj^ + CB2 — CB^ CB2 , 



(9) 



which is the CB-based "transfer function" of the check node. 
Since CBchk is a convex function of CBi, this maximiza- 
tion/minimization problem is straightforward. The maximiz- 
ing distribution dP*{p) is obtained by letting all probability 
weights concentrate on both extreme ends p = Q and p = 1/2, 
that is 



1 ^ CBin^l 

dP*{p) = <j 





if ]j = 
if p= 1/2 . 
otherwise 



Note: dP* is a probabilistic combination of a noise-free 
channel and a noisiest channel with output completely inde- 
pendent of the input, which corresponds to a BEC with erasure 
probability e = CBin.i- One visualization of this maximizing 
solution can be obtained by fixing CB2 and plotting all 
possible values of {CBi, CBchk) on a two dimensional plane, 
which form a convex curve. Connecting a string between 
both ends gives us the upper part of the convex hull of 
{CBi^CBchkY^- Therefore, the probabilistic combination of 
any {CBi, CBchk) lies within the convex hull and is upper 
bounded by the string. Since dP*{p) is a probabilistic weight 
such that the averaged CBi equals the constraint CBin.i and 



the averaged CBchk touches the string, dP*{p) must be a 
maximizing solution. 

By Jensen's inequality, the minimizing distribution dP^{p) 
is obtained by letting all probability weights concentrate on a 
single point with the same CBin.i, that is 



dP\p) 



1 if CBi{p)^2^p{l-p) = CB,n,i 
otherwise 



Note: dP^ corresponds to a BSC. 

The same arguments can be applied to find dQ* and dQ^ . 
By replacing both dP and dQ in with the maximizing 
dP* and dQ* , we prove that for general constituent BI-SO 
channels, 

CBout < CBinS + CBin^2 — C Bjn^lC Bin^2- 

By replacing both dP and dQ in ([7} with the minimizing dP^ 

and dQ^, we also have 



CBo„, > ^CBf^ ^ + CBf^ 2 CBt^ ,CBt^ 2- 

By a straightforward extension to check nodes of higher 
degree dc > 3, a similar upper bound can be obtained by 
replacing all {dc — 1) constituent channels^ with BECs having 
the same values of CBin^i- The resulting upper bound is 



CBout < 1 - n (1 " (^B,n,^). 



(10) 



i=l 



A similar lower bound can be obtained by replacing all (dc — 1) 
constituent channels with BSCs having the same values of 
CBin^i- The resulting lower bound is 



CBout > 



1 - n (1 - cBi^^). 



(11) 



'a different bounding method is to represent a check node with dc > 3 
as a concatenation of {dc — 2) degree 3 check nodes and iteratively apply 
the bound derived for dc = 3, the resulting bound of which is strictly looser 
than the bound constructed by direct replacement. For instance, [15] iteratively 
bounds the entropy by concatenating many degree 3 nodes, while [16] takes an 
approach similar to that in this paper and replaces all channels simultaneously, 
which results in tighter upper/lower bounds. 
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2) Variable Nodes: For a variable node as shown in 
Fig. |5(c)| the problem of finding an iterative upper/lower 
bound can be cast as an optimization problem as follows. 

max or min CBout = J CByar{p, q)dP{p)dQ{q) 

subject to CBin^i^ J CBi{p)dP{p) 

CB,„^2 - J CB2{q)dQ{q), 

where CByar{p,q) denotes the value of CB for the tree- 
like variable node channel if both the constituent channels are 
BSCs with parameters p and q respectively. By the definition 
of CB in Q, we have 

CBvarip,q) 



P(X = x) pfei.yak) 



p{x\y\,y2) 
p{x\yi,y2) 



(1 - p)q 



a-p)q 



pq 



■■ 4^/p{l-p)q(l - q), 



(12) 



Omitting the input arguments p and q, CB^ar can then be 
rewritten as 



CByar — CB1CB2 



(13) 



which is the CS-based "transfer function" of the variable 
node. Since CByar is a concave^ function of CBi, this 
maximization/minimization problem is straightforward and 
similar to the check node case. By Jensen's inequality, the 
maximizing distribution dP*{p) is obtained by letting all 
probability weights concentrate on a single point with the same 
CBin i, that is 



dP*{p) 



1 if CBi(p) = 2^p{l-p) = 
otherwise 



which corresponds to a BSC. The minimizing distribution 
dP^ {p) is obtained by letting all probability weights concen- 
trate on both extreme ends p ^ and p ^ 1/2, that is 

l-CB,;„,i ifp = 
dPHp)^ {CB,„,i Hp =1/2, 
otherwise 

which corresponds to a BEC. As a result, by replacing all 
constituent BI-SO channels with BSCs having the same values 
of CBinj, we obtain an upper bound for the variable node; 



(14) 



^Actually CByar is a linear function of CBi . The reason we still view it as 
a concave function is to keep the argument reusable when we are considering 
other types of noise measures (eg., SB and the conditional entropy). 



By replacing all constituent channels with BECs having the 
same values of CB, 



variable node: 



we obtain a lower bound for the 



CB, 



> n 

i=i 



(15) 



3) Combined Results: Consider BI-SO channels and the 
irregular code ensemble with degree polynomials A and p. 
By combining ( I10> and (I14> and averaging over the degree 
distributions, we have 



(16) 



where (75''-* is the value of CB after I iterations, namely, the 
value of CB for the support tree of depth 21. This is the result 
of Khandekar et al. in [14]. 

By combining il l\ and ( I15t and averaging over (A,p), we 
have a new iterative lower bound. 

Theorem 1: For BI-SO channels. 



1 



(1 - (CB('))2) 



(17) 



As stated in Sectionlllll-A. by checking whether CB'-^'' con- 
verges to zero by (I16> or by ( I17> . one can derive a lower/upper 
bound of the decodable threshold based on the (75'"' of the 
channel of interest. Closed form solutions for those thresholds 
can also be obtained following similar approaches as in [12], 
[16]. 

Similar arguments can be applied to other types of noise 
measures. In each iteration, replacing constituent channels of 
a check node with BECs/BSCs having the same value of 
SB, and replacing variable node constituent channels with 
BSCs/BECs having the same value of SB, we can reproduce 
the iterative upper/lower bound on SB found in [13]. By 
considering the conditional entropy instead of SB, we can 
reproduce the iterative upper/lower bound on the mutual 
information found in [15], [16]. 

This paper will focus on developing new bounds or strength- 
ening existing bounds for the cases in which the simple 
convexity/concavity analysis of the transfer function does not 
hold. 

IV. Z^LDPC Codes 

A. Code Ensemble 

The -based LDPC code ensemble can be described as 
follows. The value of non-zero entries in the parity check 
matrix H are limited to one, and the random parity check 
matrix ensemble is identical to the ensemble of binary LDPC 
codes introduced in Section Hl-DI The only difference is that 
the parity check equation Hx = is now evaluated in Z„i. A 
further deviation from the binary code ensemble is the Gf{q)- 
based code ensemble. Besides evaluating Hx = in GF{q), 
the non-zero entries in H are uniformly distributed between 
{1,2,-- - ,q— 1}. Further discussion of the Z,„ and GF((7) 
LDPC code ensembles can be found in [18], [24]. 
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B. Iterative Bounds 

1) Variable Nodes: As discussed in Section lTll-AI we focus 
on a variable node with degree c?i, = 3 as in Fig. |5(c)| We will 
first consider p and q being fixed (non-random) parameters 
and then extend our analysis to accommodate the random 
parameter generators dP{p) and dQ{q). 

By grouping the outputs Yi and Y2 into a 2-dimensional 
vector Y = [Yi, Y2), the variable node becomes a Z„i ^-^ 
channel, and it can be verified by definition that it is still 
symmetric. By definition (0}, the resulting CB^ari^ ^ x) 
for the vector output channel with uniformly distributed^ X 
becomes 



\/PyiPyi-x 




CB,nAO 



A compact vector representation using the component-wise 
product "•" then becomes 



CB,, 



CBi • CB2 



By iteratively applying the above inequality for variable nodes 
with dy > 3, we have 



CB^Q 



n 



(18) 



where the Y[ represents the component-wise product. Consider 
general MI-SO constituent channels with random parameter 
generators dPj{p^), where denotes the parameter vector 
for the j-th constituent channel and its distribution is denoted 
as dPj{-). Since the parameter vectors p-' are independently 
distributed for different values of j, the probabilistic average 
of the product in ( I18> is the product of individual averages. 
This implies that (I18> holds for general MI-SO channels as 
well. 

2 ) Check Nodes: Consider a check node with degree dc = 
3, namely, two constituent MSCs with parameters p and q, as 
illustrated in Fig. |5(d)| By definition, CBchk{0 x) for the 
Z,„ 1-^ Zf^ channel is given as follows: 




J2 Py^'iy^ 

Vi+y2=w+x 



m — l 

E 

w — 



J \ I 2^ PyiQw~yi 
\ \yi=a 



' m- 



^ ^ PyiQx+w-yi 

\yi=o / 



(19) 



'By the lineaiity of the Zm parity check code, the marginal distribution 
of any Xi,Wi G {1, ■ ■ ■ , n} is either uniform or concentrated on {0}. The 
latter case is of little interest since those bits can then be punctured without 
affecting the performance. 



Each summand in (I19> can be upper bounded by 



\ 




^ ] PyiQx+w-yi 

yi=o 



\ 




m—l m—l 



A ^ ] ^ ] PyQw — yPzQx+w — z 
\ y=o z=0 

m— 1 7n— 1 

< E E VPyPWl^-v1^+^- 

y = z=0 



(20) 



where the inequality follows from the fact that Xi < 
J2 \fxi if Xi > 0, Vi. By combining ( I19l l and (I20> . we have 



CB,hk{0^x) 

m—l m—l m—l 

- E E E VPyPW<lw-y1x+w-z 

w=0 y=0 z=0 
m—l m—l m—l 

= E E E y/PyPy+=^Wl^"l^'+^-^' 

w'=0 y=0 2'=0 
m—l 

= Y CB.nAO ^ z')CB,nMO -^x- z'), 



(21) 



z'=0 



where (12 1> follows from the change of variables: w' = w — y 
and z' ^ z — y. A compact vector representation using circular 
convolution "igi" then becomes 



CBchk < CBi. 



CB; 



(22) 



By iteratively applying the above inequality and noting the 
monotonicity of the convolution operator (given all operands 
are component-wise non-negative), we have the following 
inequality for the cases dc > 3: 



CB 



chk 



< (g) CB™,,. 

1=1 



(23) 



Since the circular convolution is a summation of products and 
the p^ are independently distributed for different values of j, 
the probabilistic average of the circular convolution in ( I23> is 
the circular convolution of individual averages. This implies 
that i23i holds for general MI-SO channels as well. 

Note: (I22> is loose for the binary case (m = 2). For 
TO > 2, there are many non-trivial cases in which i22i is 
tight. For example, suppose m = 6, p^ = (0.5,0.5,0,0,0,0), 
and p2 = (0.5,0,0.5,0,0,0). We have CB^hk = 
(1,0.75,0.5,0.5,0.5,0.75), CB„a (1,0.5,0,0,0,0.5) and 
CBi„.2 = (1,0,0.5,0,0.5,0), which attains the equaUty in 
(I23. 

3) Combined Results: Consider general MI-SO channels 
and the irregular code ensemble with degree polynomials A 
and p. By combining ( I18> and i23\ and averaging over the 
degree distributions, we have proved the following theorem. 
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Theorem 2: Let CB*^'^ denote the value of CB for the 
support tree channel after I iterations. Then we have 

Cb('+i)<CB(")a(p(cB«)), (24) 

where the scalar products within A(-) are replaced by 
component-wise products, and the scalar products within /?(•) 
are replaced by circular convolutions. 

For a lim code ensemble with degree polynomials (A,/?), 
we can first fix an arbitrary** CB*, let CB^"^ = CB*, and 
iteratively compute the upper bound by (I24> ^. Suppose that, for 
all X ^ 0, limj^oo CB^^\Q x) = 0. By Lemma^ any MI- 
SO channel with CB < CB* is guaranteed to be decodable by 
the BP algorithm when sufficiently long codes are used. Unlike 
the two-dimensional case, the thresholds determined herein 
for in > 2 cases do not admit straightforward closed form 
solutions due to the lack of ordering. Some further research is 
necessary to determine the closed form threshold "envelope" 
of the decodable CB vectors. 

C. Stability Conditions 

The sufficient stability condition for Z,„ LDPC codes can 
be obtained as a corollary to Theorem^ 

Corollary 1 (Sufficient Stability Condition): Consider any 
MI-SO channel with noise measure CB and any LDPC 
code ensemble with degree polynomials (A, p). If 

X2p{l)CB{0^ x) <1, V2;eZ™\{0}, 

then this Z„i code is stable under the BP decoder. Namely, 
there exists e > such that if after Iq iterations, 
max^g2;„\{o} CB('")(0 x) < e, then lim;_,oo CB(')(0 
x) = for all X 7^ 0. (Or equivalently lim;„too pi'' = 
0.) Furthermore, the convergence rate of CB^^\Q x) is 
exponential or superexponential depending on whether A2 > 
or A2 = 0. 

Proof: Define = max^gz„\{o} CS(')(0 ^ x). We 
prove the following equivalent statement that there exist e, 5 > 
such that if /^^ G (0, e) for some lo, we have 



JCB 

Ai) 

JCB 



<\-6, yi> Iq. 



(25) 



Without loss of generality, we can assume /^'^ = e. Using 
the fact that CB(')(0 -> 0) = 1 for all I e N, and the 
monotonicity of the convolution operator when all coordinates 
are possitive, it can be shown that Vx £ Z,„\{0}, 

(CB^'^Y'^^' {0^x)< ((4 - l)e + 0{e')), 

where "(g)" represents the convolution product. Similarly, for 
the component-wise product, one can show that 



x) < e^" 



^When uniform a priori distributions on X are considered, any valid CB 
must satisfy the symmetric condition in js) and that CB{0 x) G [0, 1], 

During the iterations, we may further strengthen ^ by CB('+^) < 
min 1 1, CB'"^ A ^p^CB^^^j, since any valid CB value is upper bounded 
by 1. 



Using the above two inequalities and J24> . we have 

CB^^+^\Q^x) 



k-l 



< CB(^^ (0 ^ x) ^ A J ^ pn{h - l)e + 0{e^) 

k \ h I 

= CB(0)(0^x)A2p'(l)e + O(e2). (26) 

Since \ip' (X)CB(fd ^ x) < l,Vx G Zm\{0}, we can choose 
a 5 > such that 1 - 5 > \2p'{X)jc\. With a fixed choice 
of (5, ( I25t is satisfied for sufficiently small e. J26t also shows 
that 



lim 



JCB 



ICB 



fPB^2p'{l). 



(27) 



Hence the convergence rate is exponential or super exponential 
depending on whether A2 = 0. The proof is thus complete. ■ 
A matching necessary stability condition can be proved as 
follows. 

Theorem 3 (Necessary Stability Condition): Consider any 
MI-SO channel with noise measure CB and any Z^ LDPC 
code ensemble with degree polynomials (A, p). If 

3x0 e Z„A{0}, such that A2p'(l)CS(0 ^ xq) > 1, 

then this Z,„ code is not stable under the BP decoder. Namely, 
there exists xq G Zm\{0} such that Mmi-foo CB^'^^O 
Xq) > 0, or equivalently, limi_>ooPe'' > 0. 
A detailed proof using channel degradation argument similar 
to [6], [18] is provided in APPENDlxlTTTl 

We close this subsection by showing the stability results for 
GF{q) LDPC codes in [18] can be derived as a corollary to 
the above stability conditions for Z„i LDPC codes. 

Consider a MI-SO channel with noise measure CB, and 
a GF(g)-based LDPC code with degree polynomials (A, p), 
where g is a prime number. The following stability conditions 
for GF(q) LDPC codes can be derived as direct corollaries 
to Corollary and Theorem which were first presented in 
[18]. 

Corollary 2 (Sufficient Stability Condition): If 

,I]xeGF(g)\{0} CB{Q - 



A2P'(1)- 



1 



<1, 



then this Gf{q) code is stable under the BP decoder. 
Corollary 3 (Necessary Stability Condition): If 

. J2xeGFiq)\{a} CB{0 - 



A2P'(1) 



9-1 



>1, 



then this Gf{q) code is not stable under the BP decoder 

Since the stability conditions of the Z,„ LDPC codes rely 
only on the pairwise error probability and the multiplication 
of the uniformly distributed edge weight w G GF(g)\{0} is 
equivalent to a uniform permutation of all non-zero entries, 
the stability conditions of a Gf{q) code are equivalent to 
those of a Z^ code with the pairwise "error pattern" averaged 
over all non-zero entries. As a result, all results for Z,„ 
codes involving only CB{Q x) hold for Gf{q) codes as 
well with each CB{0 x) being replaced by the average 
Ci3(0 x). The above corollaries then become 
simply the restatement of the stability conditions for Z^ 
LDPC codes. 
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D. Applications 

We close this section with a non-comprehensive list of 
practical applications based on higher order LDPC codes and 
some corresponding references. 

1) Improving the code performance [25]: By grouping two 
bits into a GF(4) symbol, the finite length performance 
of codes can be enhanced by the resulting higher dimen- 
sional codes. 

2) Higher order coded modulation. Berkmann used higher 
order codes for coded modulation with the natural code- 
to-symbol mapping [26], [27]. By exhaustively search- 
ing for the optimal code-to-symbol mapper over all 8! 
possible mappers, our simulation shows that the asymp- 
totic threshold of the regular Zs-based (3,9) code can be 
improved to within 0.63dB of the channel capacity of the 
8PSK constellation with no additional cost. The simple 
structure of the (3,9) code and the better word-error 
probability make it an appealing alternative to turbo 
coded modulation or LDPC-coded bit-interleaved coded 
modulation (BICM). 

In addition to having lower decoding complexity, one 
argument that the BICM is favorable over Z„i-based 
coded modulation is the higher cutoff rate of BICM 
[23], which leads to a common belief that it is easier 
to design good codes for BICM than for high order 
coded modulations. The higher cutoff rate of BICM can 
be better explained by the fact that the origin of the 
cutoff rate can be traced to the pairwise union bound 
(the Gallager bound) over different symbols, which 
results in the summation operation over all possible 
CB{0 x) values in (|6}. Our stability conditions 
show that the "effective" value of CB (in terms of the 
code performance) is the maximum of all CB{Q ^ x) 
rather than their summation. This result demonstrates 
that the cutoff rate, involving the sum rather than the 
maximum of CB{Q x), is not a good benchmark 
between channels with different orders of alphabets. 
The above argument also gives a partial reasoning of 
the performance improvements after moving to a higher 
dimensional code in terms of the effective CB value, 
since the max CB{Q x) can be reduced by a good 
coded-alphabet to transmitting-symbol mapper, and a 
greater stability region can be obtained. 
Note: Another advantage of the higher order coded 
modulation over BICM is that a matched spectral null 
code can be concatenated as an inner code to improve 
the performance when in inter-symbol interference (ISI) 
environments [21]. 

3) Constructing mutual-information-achieving codes with 
non-uniform coded bit distribution by converting Z,„ 
codes with uniform symbol distributions into non- 
uniformly distributed binary codes using "symbol map- 
pers" [28]. Among the applications here are the follow- 
ing. 

• Constructing codes for cases in which the capacity- 
achieving a priori distribution is not uniform [28], 
[24]. 



• Designing optimal superposition codes for broad- 
casting channels. 

* Designing codes for optical channels with cross talk 
[29]. 

Other references on higher order LDPC codes can be found 
in [30], [9] 

V. Iterative Bounds on Binary Channels 

In this section, we will first show that the existing CS-based 
iterative bounds for BI-SO channels also hold for BI-NSO 
channels. Then we will strengthen the existing CS-based and 
5i3-based bounds by providing a two-dimensional {CB, SB)- 
based iterative upper bound for BI-SO channels. 

A. CB-based Bounds on BI-NSO Channels 

The definition of CB in Q is applicable to BI-NSO chan- 
nels with either uniform or non-uniform prior distributions. 
For the following, we will show that the inequalities ( I16t and 
(I17> hold for BI-NSO channels as well by assuming a uniform 
prior distribution on X = {0, 1} and by adopting the reverse 
channel perspective. A uniform prior distribution is commonly 
assumed in all existing work on iterative bounds of LDPC code 
performance [13], [14], [15], [16], which can be justified by 
the perfect projection condition in [31]. 

Theorem 4 (CB-based Bounds for BI-NSO Channels): 
For the irregular LDPC code with degree polynomials (A, p), 
the iterative upper and lower bounds ( I16t and ( I17> hold for 
BI-NSO channels. 

Proof: We will first focus on the simplest binary- 
input/binar-ouput non-symmetric channel, which is illustrated 
in Fig. |6(a)| and is denoted as BNSC (in contrast to BSC). 
Since any BI-NSO channel can be regarded as the probabilistic 
combination of many BNSCs, our results for BNSC can then 
be generalized to arbitrary BI-NSO channels. 

Any BNSC can be specified by two scalar parameters po^i 
and pi^o, where Pz^z' denotes the conditional probability 
9{Y ^ z'\X = z). CB thus becomes 

CB = v/po^i(l -Pi^o) + v/pi^o(l-po^i). (28) 

We can also represent this X ^ Y BNSC from the reverse 
channel perspective F i-^ X as in Fig. |6(b)| such that 



7-0^1 



1 


- Po^i + 


Pl^O 




Pa^i 




1 


+ Po^i - 


Pl~*0 


1 


- Po^i + 


Pl^O 




2 




1 


+ Po^i - 


Pl^O 



where Vz^z' = z'\Y = z) and R{y) = P{Y = y). 

Then by definition, we have 

= i?(0)2Vro^i(l - ro^i) + i?(l)2 Vri^o(l - n^o) 
= R{Q)CB{ro^i) + R{l)CB{n^o), 
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P{X = 0) = 0.5 


BNSC 

1 -Po^i 




PiX 1) = 0.5 








1 -pi^o 





(a) Nonnal Perspective. 





BNSC 

1 - ro^i 


P(Y ^0)^ R{0) 






p(y = i) = i?(i) 




1 - ri^o 





then conditioning on the event {Xi = X2}- From the above 
equation, we can further simplify CByar as follows. 



CB„ 



E E 

!;ie{o,i}!y2e{oa} 
•2, 



Riyi)S{y2)P jXi = X2\Y ^ iyi,y2)) 

PiXi=X2) 

' P{X,=X2 = 0\Y ^{y,,y2)) 
P{X,^X2\Y = {yuy2)) 

/ p(Xl=X2 = l|Y=(z/i,y2)) 

Y P(Xi =X2|Y= (yi,2/2)) 

E ^(yi)V 

\vie{o,i} 



(b) Reverse Perspective 
Fig. 6. The probabilistic model of the BNSC. 



E 



in which CB{p) = 2-\/p(l — p) computes the value of CB 
for a BSC with crossover probability p. This representation 
separates the previous entangled expression of CB in ( I28> 
so that it is as if there are two BSCs with parameters ro^i 
and ri_>o respectively, and these two BSCs are probabilis- 
tically combined with coefficients i?(0) and We use 
{r..}) to represent this reverse channel perspective. 
Consider the variable/check nodes of degree 3 with two 
constituent BNSCs in reverse form, namely, Chl:(i?(-), {r..}) 
and Ch2:(S'(-), {s -}), such that 

CB.^s := Riyi)CB{ry,^o), 
yie{o,i} 

CB,n,2 := Siy2)CB{sy,^o). 
y26{o,i} 

For a variable node of degree dy = 3, by definition and after 
some simple algebra, we have 



CBya 



-X.Y 



\ p{X\Y^,Y2) 
p{X\Y^,Y2) 



-X\Y^.Y^2 



\ p{X\Y^.Y2) 
p{X\Y^.y2) 



(a) 



J2 J2 p(Y = (2/1,2/2)1x1 = X2 

:e{o,i}!/ie{o,i}y2e{o,i} 



• VPiXi^X2=x\Y = iyi,y2),Xi=X2) 

• VP (Xi -^2 =5|Y = (2/1,2/2), Xi =^2). 

where Xi and X2 are the individual inputs of Channels 1 
and 2. The last equality follows from the fact that the probabil- 
ity distribution with a variable node constraint X = Xi = X2 
is identical to the conditional distribution by first assuming Xi 
and X2 are i.i.d. uniform Bernoulli distributions on {0, 1} and 



+0Sl/2^1 

V2/2e{oa} 

= CBinS ■ CBin^2, (29) 

By noting that ( I29> possesses the same form as in il3\ . all our 

previous analyses for variable nodes with BI-SO constituent 

channels hold for BI-NSO channels as well. 

C onsid er a check node of degree dc ~ 3, which is similar to 
Fig. |5(d)| except that the constituent channels are now BNSCs. 
By aennition, some simple algebra, and the observation that 
X = Xi + X2, we have 

CBchk 

= E E P(Y = (yi,j/2)) 
Hie{o,i} !/2e{o.i} 

■2\/P(Xi +X2 



E E 

!/l6{0,l}!/2e{0,l} 



= 0|Y = (yi,j/2))P{Xi 

R{yi)S{y2) 



-X2 = l|Y = fei,J/2)) 



• v'4fi,i^or-Hi^i +4sj;2_oSi;2_i - 16rj;i_orj;i_iSy2_oSj/2^i 

= E E ^iy^)S{y2) 
!/ie{o,i} i;2e{o,i} 

(30) 

Note that ( I30t possesses the same form as in and (|9}. Thus, 
each BNSC in i30\ has the same effect as a BI-SO channel 
corresponding to a probabilistic combination of two BSCs 
with parameters ro_»i, ri^o (or so_ti, si^o) and weights 
R{0) and R{1) (or S'(O) and S'(l)). Since ^ and ([TTll hold 
for general BI-SO channels, they also hold for this particular 
combination of two BSCs, which in turn implies that they hold 
for BNSCs as well. By taking the probabilistic combination 
of many BNSCs, we have shown that (llOt and (^J hold for 
general BI-NSO channels as well. 

Since our previous analyses for both variable and check 
nodes (with BI-SO channels) hold for BI-NSO channels as 
well, we have proved Theorem |4| ■ 

B. A Two-Dimensional Upper Bound on BI-SO Channels 

In this section, we develop a two-dimensional upper bound 
on the {CB, SB) pair of a BI-SO channel, for which convex- 
ity/concavity analysis of the transfer function is not sufficient. 
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Similar to the one-dimensional results in Section Illl-BI we 
consider variable node and check nodes separately. 

1 ) Check Nodes: Suppose the check node channel has two 
constituent BSCs with crossover probabilities p G [0,1/2] 
and q G [0, 1/2] as shown in Fig. |5(d)| where p and q have 
distributions dP{p) and dQ{q), respectively. Let CBin.i and 
SBin^i denote upper bounds on the values of CB and SB 
for the first constituent channel and let CBin,2 and SB in ^2 
denote corresponding upper bounds for the second constituent 
channel. We would like to develop an upper bound on the pair 
{CB, SB) for the support tree channel. This iterative bounding 
problem thus becomes: 



max CBout^ j CBM{p,q)dP{p)dQ{q) 
SBout = j SB,Hk{p,q)dP{p)dQ{q) 
subject to j 2^Jp{l -p)dP{p) < CBin,i 
J Ap{l~p)dP{p) < SB„,,i 



(31) 



2^q{l - q)dQ{q) < CB,„,2 
J 4q(l - q)dQ{q) < SB,„,2, 



where CBchk {p, q) is defined in (|8} and 



SB. 



chk 



4p{l -p)+ 4q{l -q)- 4p(l - p)4q{l - q). 



Using some simple algebra, we can show that the opti- 
mum value SB*^f. satisfies SB^^t = S'-Bm.i + SBin,2 — 
SBin,iSBin,2- The remaining problem reduces to the max- 
imization of CBout subject to two input constraints on each 
of dP and dQ. Solving this optimization problem, the maxi- 
mizing dP* and dQ* can be expressed as follows. 



dP*{p) 



where t 



1 - 

CB, 



CBi. 



if p = 





SBin^l 
CBin.i 



if 2^/^(1 -p) 
otherwise 



dQ* can be obtained by replacing CBin^i, SBin,i, and p in 
the above equation with CBin,2, SBin.2, and q, respectively. 
A proof of the optimality of dP* and dQ* is given in 
AppendixITvI 

By substituting all constituent BI-SO channels with chan- 
nels of the same form as dP*, we obtain an upper bound on 
{CB, SB) in check node iterations as follows. 

Theorem 5 (U Bcb,sb in Check Node Iterations): 
Suppose the check node degree is dc and the input 
{CB, SB) pair is upper bounded by {CBin, SBin). Then the 
pair {C Bout tS Bout) of the check node iteration is bounded 



by 



SBout < 1 ^ (1 ^ SBin) 



CB, 



out 



i=l 



4-1 

i 



\ 



1-1- 



SB,, 
CBi 



1 - 



CBl 



CB 



SBin J \y SBin J 

Corollary 4: For the check node iteration of any (A, p) 
irregular LDPC codes, we have 



SBo 



CBo 



< l-p{l-SB,n) 



< 



k-1 



k 



\ 



1 - 



S Bir. 



fc-1- 




V ^Bin , 

Note: By incorporating the SBin constraint, the CB bound 
( I32t is now tight for both the BEC and BSC cases, which is a 
strict improvement over the CS-only bound (I10> . (The bound 
jlO> is obtained by connecting the two ends of the CS-based 
transfer function curve and is tight for the BEC case but loose 
for the BSC case.) 

2) Variable Nodes: We consider a variable node of de- 
gree dv = 3. Given that the {CB, SB) values of the con- 
stituent channels are upper bounded by {CBin.i, SBin.i) and 
{CBin.2, SBin.2), respectively, the iterative upper bounding 
problem becomes 



max CBo 



CBuarip,q)dP{p)dQ{q) (33) 



SBout = J SB,,ar{p,q)dP{p)dQ{q) 
subject to j 2^p{l - p)dP{p) < CBin,i 

J Ap{\~p)dP{p) < SB,n,l 

J 2^q{l - q)dQ{q) < CB,n,2 
j 4q{l-q)dQ{q) < SB,n,2, 
where CBuar {p, q) is defined in M2\ and 



SByar{p,q) 



Ap{l - p)Aq{l ~ q) 



Ap{l-p){l-4q{l-q))+4q{l-q)- 



By some simple algebra, it can be shown that the optimum 
value CB*^f. satisfies CB*^f. ~ CBin.1CBin.2- Unfortunately, 
for the remaining maximization problem on SBout, the max- 
imizing distribution dP* depends on both {CBin^i, SBin,i) 
and {CBin.2, SBin,2)- The simple replacement of each con- 
stituent channel with a maximizing counterpart does not 
work this time. To circumvent this difficulty, we provide 
an upper bounding distribution dP** depending only on 
{CBin.i, SBin.i), such that the objective value of any feasible 
solutions dP and dQ is no larger than the objective value ob- 
tained from dP** and dQ. The distinction between the upper 
bounding distribution dP** and the maximizing distribution 
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dP* is that dP** may not be feasible and thus may serve 
merely the bounding purpose. 

For simplicity, we express dP** by dropping the subscript 
1 in the vector constraint {CBins, SBin,i)- 



dP**{p) 



where 




Isb) 



t+CBi. 



xf2^p{l-p)=CB,, 



CB,; 



SB) t+CBi. 



if 2 ^p(l-p) = 

if 2v/p(l-rt =t 
otherwise 



(34) 



which satisfies CB < 




2y/ SBin — 

otherwise 



t ^ 
> 



+ {t- CB.nYw, 



(35) 



if i{2^SB~) < 
otherwise 

The upper bounding distribution dQ** for the second con- 
stituent channel can be obtained by symmetry. A derivation 
of dP** is included in APPENDIX |y] It is worth noting 
that when there is no constraint on CBin (namely, when 
CBin ~ \/SBin by Lemma 0, dP** collapses to a BSC, 
which coincides with the S'i?-based bound in [13]. Hence the 
upper bound distribution dP** is a strict improvement over 
the existing SB-based bound. 

Using this upper bounding distribution dP** , an upper 
bound for (CB, SB) for variable node iterations is given as 
follows. 

Theorem 6 (UBcb,sb in Variable Node Iterations): 
Suppose the variable node degree is d^, the input {CB, SB) 
pair is upper bounded by {CBin, SBin), and the uncoded 
channel has noise measures {CBq, SBq). Then, the output of 
the variable node iteration is upper bounded by 



CBout < CBo{CB 



and 



SBout < ^d^-l {{CBq, SBo), {CBin, SBin)) , 

where $(i„-i computes the value of SB for a variable node 
channel with one constituent Cho channel and {d^ ~ 1) 
constituent Chin channels. Here, Chin and Cho are of the 
form of dP** and can be uniquely specified by {CBin, SBin) 
and {CBq,SBq) respectively. 

Corollary 5: For the variable node iteration of any (A, p) 
irregular LDPC codes, we have 

CBout < CBoX{CBin) 

SBout < 5]pfc$fc-i((CBo,SBo),(CB,„,5B„)). 

k 

An explicit expression for involves a direct sum of 

various terms, the complexity of which grows at the order 
of k^. A more practical, fast implementation is via the fast 
Fourier transform (FFT), which is similar to that used in 



density evolution. We first calculate the LLR message distri- 
bution dP{m) for Cft-o and Chin from the upper bounding 
distribution dP**{p) in (l34l . Since the output LLR is the 
summation of input LLRs, the distribution of the output LLR is 
the convolution of the input LLRs, which can be calculated by 
FFT. At the end, we can use Q to compute the corresponding 
output SB value. 

3} Two-dimensional Iterative Upper Bound U Bcb.SB- 
By combining the aforementioned upper bounds for the vari- 
able node and the check node iterations, we obtain a two- 
dimensional iterative upper bound UBcb.sb- Since this two- 
dimensional bound is based on separate analysis of variable 
nodes and parity check nodes, it can be applied to any LDPC- 
like codes with graph-based ensembles without modification, 
including regular/irregular repeat accumulate (RA) codes [4], 
and joint-edge-distribution LDPC codes [32]. 

We omit the explicit expression for this two-dimensional 
bound since it is a direct concatenation of Theorems ]5\ and 
|6] By iteratively computing the upper bound {CB^^\ SB^'-^) 
and testing whether it converges to (0, 0), we can lower 
bound the decodable threshold for general BI-SO channels. 
The performance comparison of this procedure to existing 
results will be discussed in Section fvTll 



C. Some Notes on Searching for High-Dimensional Bounds 

Under the framework proposed in the previous sections, 
the problem of constructing iterative upper/lower bounds is 
equivalent to solving a corresponding optimization problem, 
within which the "variables" correspond to the probabilistic 
weight, dP{p), of the corresponding BI-SO channel. Since 
most of the common noise measures of a BI-SO chan- 
nel can be computed by the probabilistic average over the 
corresponding measures of the consitituent BSCs, both the 
constraints and the objective functions are generally linear 
with respect to dP{p). The optimization of interest becomes 
a linear programming (LP) problem, of which the methods 
of finding optimal solutions ares well studied. Two notes 
about searching for upper/lower bounds are worth mentioning. 
First, when considering high-dimensional objective functions, 
the corresponding LP problem generally does not admit a 
uniform optimizer, as shown in our {CB, SB) analysis in 
the previous sub-section, which hampers the use of simple 
channel replacement for bounding purpose. Second, the closed 
form solutions become more and more difficult to obtain when 
complicated constraints are applied, as demonstrated in ( I34> . 
An alternative route is to use a commercial LP solver to 
numerically find bounds for each iteration. The closed form 
solution on the other hand is computationally much more 
efficient and provides better insight when compared to the 
numerical method. Since an iterative upper bound guarantees 
the minimum decodable threshold and is of higher importance 
from both the practical and theoretical perspectives, we use 
the two-dimensional upper bound to demonstrate this new 
framework and leave the two-dimensional lower bound for 
future research. 
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VI. A One-Dimensional Non-Iterative Bound on 
BI-SO Channels 

In this section, we construct a non-iterative upper bound, 
which is the best known bound that is tight for BSCs. 

First we introduce some new notation. Letpe^ denote the bit 
error probability of the belief propagation after I iterations. To 
distinguish between the types of BI-SO channels on which 
we are focusing, we append an argument Fc to the end 
of pe\ That is, Pe\Fc) denotes the bit error probability 
after I iterations with the conditional distribution of the BI- 
SO channel being Fc {f{y\x)}. In a similar fashion, 
we define CB^'^Fc) as the Bhattacharyya noise parameter 
after I iterations with the BI-SO channel being Fc, and 
SB^'''>{Fc) is defined similarly. Following this definition, 
CB{Fc) := CB'-°^Fc) denotes the Bhattacharyya noise 
parameter of the uncoded BI-SO channel Fc- For simplicity, 
we use Fbscp denote the Fc of a BSC with crossover 
probability p, and similarly we define Fbec.h- Suppose for 
some Fsscp, '^^e LDPC code ensemble is decodable. By the 
channel degradation argument in [6], one can show that all 
BI-SO channels Fc with p'-e\Fc) < p are also decodable, a 
formal statement of which is as follows. 

Theorem 7 (The Channel Degradation Argument in [6]): 
Suppose Fc is a BI-SO channel and Fbsc,p is a BSC. If 
Pe{Fc) = Pe{FBSC,p), then for any I e N and any irregular 
(A, p) LDPC codes, 

pl^\FBsc,p)>pi'\Fc). 

The above inequality holds as well when subsituting pi' ^ (•) by 
other common noise measures including CSW(-), 
and the conditional entropy. 

This result, though being tight for BSCs, generally gives a 
very loose bound for other channels. We strengthen this result 
by providing a strictly tighter bound in the following theorems. 

Theorem 8: Suppose Fc is a BI-SO channel and Fbsc.p 
is a BSC. If SB{Fc) = SB{Fbsc,p), then for any / € N and 
any irregular (A, p) LDPC codes, 

CB^'\Fbsc,p)>CB^'\Fc). 

In Theorem^ it is possible that pT' {Fbsc,p) < Pe\Fc), 
which is different from the result using the channel degradation 
argument. 

Corollary 6: If a (A, p) irregular LDPC code is decodable 
for a BSC with crossover probability p*, then any BI-SO 
channel Fc with SB{Fc) < SBiFBsc^p*) = 4p*(l - p*) 
is decodable under the same (A, p) code. 

Proof: For any symmetric channel Fc with 
SB{Fc) < SB{Fbsc.p*), we consider a Fbsc.p such 
that SB{Fbsc.p) = SB{Fc). Since Fbsc,p' is physically 
degraded w.r.t. Fbsc.p, Fbsc.p is also decodable, namely, 
limi^oc Pc^Fbscp) = 0. By the relationship between pe 
and CB in Lemma\7\and by Theorem^ we have 

■2p^e HFc) < CB^'\Fc) < CB^'\Fbsc.p) 

< 2^pi'\FBsc.p)^o{l). 



This completes the proof. ■ 
Corollary |5] can be used as a tight one-dimensional upper 
bound, which is denoted by UB'gg. 

A proof of Theorem^is given in APPENDlx[^ We close 
this section by providing a lemma showing that Theorem]8\is 
a strict improvement over Theorem{7\ the channel degradation 
argument. 

Lemma 3: Suppose Fc is a BI-SO channel and Fbscp 
is a BSC. If p,XFc) = PeiFBSc.p), then SB{Fc) < 
SB{Fbsc.p)- Therefore, {Fc : SB{Fc) < SB{Fbsc.p)} 
is a super set of {Fc : Pe{Fc) < Pe{FBSC.p)}- 

Proof: Let dPpcip) denote the probabilistic weight of 
the BSCs corresponding to the BI-SO channel Fc- Since 

SB{Fc) = J Apil - p)dPFa{p), Pe{Fc) = Jp dPpcip), 

and 4p(l —p) is a concave function of p, Lemma]3\is a simple 
result of Jensen's inequality. ■ 

VII. Performance Comparisons 

In this section, we compare the tightness of various lower 
bounds on the asymptotic decodable thresholds, obtained from 
the existing results and our results of Sections IV-AI IV-BI and 

EH 

Three existing results are included in TABLE HI including 
one based on the Bhattacharyya noise parameter [14], denoted 
as UBcB, one on the soft bit value [13], denoted as UBsb, 
and one on the conditional entropy H{X\Y) [15], [16], 
denoted as UBinfo- UBcb.sb denotes the two-dimensional 
{CB. S'B)-based bound provided in Section EH and UB*sb 
denotes the non-iterative tight bound given in Section IVTI The 
DE column lists the asymptotic decodable thresholds obtained 
from density evolution [6]. In Section IV-AI UBcb has been 
generalized for arbitrary BI-NSO channels. Therefore, the non- 
symmetric z-channel"^ is also included for comparison, in 
which the asymptotic threshold is obtained from the gener- 
alized density evolution method for BI-NSO channels [17]. 

As proved in Section IV-BI and evidenced in TABLE U 
the two dimensional bound UBcb.sb provides strict im- 
provement over UBcB and UBsb- For channels that are 
neither BSC-like nor BEC-like, e.g. BiAWGNC and BiLC, 
the bound UBmfo found by Sutskover et al, is tighter than 
UBcb.sb while the two dimensional UBcb.sb is tighter at 
both extreme ends. This phenomenon can be explained by 
the convexity/concavity analysis of the transfer functions. For 
UBcB the bounding inequality resides in the check node 
iteration, in which BECs attain the equality. Therefore, U Bcb 
is the tightest when channels are BEC-like. For UBsb, the 
bounding inequality resides in the variable node iteration, in 
which BSCs attain the equality, so UBsb is preferred for 
BSC-like channels. UBcb.sb thus has better performance 
in both extreme cases. On the other hand, UB^fo invokes 
bounding inequalities in both the variable node and the check 
node iterations. We observe that the absolute values of the 
curvatures of the transfer function is generally smaller when 
expressed in terms of the mutual information. Therefore, for 
the sake of insight, better predictability is obtained when the 

'"The z-channel is a BNSC such that pi_*() > and po^i = 0- 
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Comparisons of lower bounds on decodable thiesholds. 





DE 


UBsB 


UBcB 


U Bin fa 


UBcB,SB 


UB%„ 


Uecodable 
Thrpshnlds 


— 


SB* > 0.2632 


CB* > 0.4294 


h* > 0.3644 


— 


SB* > 0.3068 


BEC (e*) 


Si 0.4294 


> 0.2632 


> 0.4294 


> 0.3644 


> 0.4294 


> 0.3068 


Rayleigh (ct*) 


« 0.644 


> 0.5191 


> 0.6134 


> 0.6088 


> 0.6148 


> 0.5804 


Z-channels (Pj^g) 


Si 0.2304 




> 0.1844 








BiAWGNC (cr* ) 


Si 0.8790 


> 0.7460 


> 0.7690 


> 0.8018 


> 0.7826 


> 0.8001 


BiLC(A*) 


« 0.65 


> 0.5610 


> 0.5221 


> 0.5864 


> 0.5670 


> 0.6146 


BSC (p*) 


Si 0.0837 


> 0.0708 


> 0.0484 


> 0.0696 


> 0.0710 


> 0.0837 



TABLE I 



Comparison of lower bounds derived from finite-dimensional iterative upper bounds. 



channel of interest is neither BSC nor BEC-like, e.g., the 
BiAWGN channel. 

By Lemma the feasible {CB, SB) pairs satisfy CB > 
SB and {CBf < SB. By plotting general BI-SO channels 
according to their {CB, SB) values, the set of decodable 
channels forms a "decodable region" and Fig. demonstrates 
the decodable region of regular (3, 6) codes. The decodable 
region is plotted with a thicker boundary using the channels 
considered in TABLEU The density evolution method does not 
guarantee that all channels with {CB, SB) inside the region 
are decodable. It is possible that two types of channels have 
the same {CB, SB) values but one is decodable while the 
other is not. 

The vertical line in Fig. marked by UBcb represents 
the inner bound of the decodable CB threshold [14]. The 
horizontal line marked by UBsb represents the inner bound 
of the decodable SB threshold. Our results on the two- 
dimensional bound and the non-iterative tight bound greatly 
push the inner bounds of the decodable region toward its 
boundary (the curve marked UBcb,sb and the horizontal 
line marked by UB*gg). These bounds guarantee that all BI- 
SO channels with {CB, SB) within the inner bounds are 
decodable under belief propagation decoding. In Section IV-AI 
we have shown that the vertical line UBcb holds as an inner 
bound even for BI-NSO channels. 

It is worth noting that all bounds for binary-input channels 
in Sections IVl and IVTI are obtained by simple channel replace- 
ment. Our proofs show that replacement of any one of the 
constituent channels will result in a new upper/lower bounding 
tree channel, and the replacement of all constituent channels 
gives us an upper/lower bound admitting closed form solutions, 
as those shown in the previous sections. In some situations, it 
is more advantageous to replace only part of the constituent 
channels, which results in tighter bounds at the expense of 
more complicated/channel-dependent solutions. For example. 
Theorem 7 in [13] provides a different iterative upper bounding 
formula by substituting all {dy — 1) C/ii„ channels of a 
variable node with the corresponding maximizing BSCs while 
leaving the observation channel CIiq intact. The result is a 
channel-dependent iterative formula with tighter performance 
than the IJ Bsb obtained by the replacement of all channels. 
The benefit of using selective channel replacement is also 
pointed out in [16]. 

A tight outer bound of the decodable region was proved 
by Burshtein et al. [13], illustrated by the horizontal line 



Decodable and Feasible Region 

Inner Bnd by UB 



0.5 - 



Outer Bnd by LB, 




0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 



CB 

Fig. 7. The decodable region of the regular (3, 6) code in the (CB, SB) 
domain and some inner bounds of the decodable region. 

marked by LBsb in Fig- Based on the mathematical 
symmetry between CB and SB in variable node and check 
node iterations, we conjecture the existence of a tight outer 
bound in terms of CB, which remains an open problem. 

VIII. Conclusions 

Finite dimensional bounds on the decodable thresholds find 
applications in both theoretical analysis and practical approxi- 
mations. In this paper, we have developed a new iterative upper 
bound for -based LDPC codes on MI-SO channels, which 
leads to a sufficient stability condition and provides insight 
into the analytical structure of general LDPC codes. Combined 
with a matching necessary stability condition proved herein, 
our stability condition pair can be used to derive the existing 
stability conditions for GF((7)-based LDPC codes. 

Two new bounds for binary codes on BI-SO channels 
have also been constructed based on two types of noise 
measures, the Bhattacharyya noise parameter CB and the soft 
bit value SB. These bounds push the existing inner bounds of 
the decodable region toward its boundary. An iteration-based 
bound for general memoryless BI-NSO channels, which finds 
applications in optical channels or magnetic storage channels, 
has also been derived. 

Throughout this paper, a new framework enabling system- 
atic searches for more finite-dimensional bounds has been 
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provided, under which we have modelled the iterative bound- 
ing problem by considering its probabilistic decomposition. 
The performance discrepancy among various bounds can be 
explained by the tightness of different bounds during the vari- 
able node and the check node iterations. Besides the implied 
uniform good performance over all types of channels, these 
new finite dimensional bounds and the proposed framework 
provide a useful tool for studying the behavior of iterative 
decoding. 

Appendix I 
Proof of the MSC Decomposition 

Proposition 1: Any MI-SO channel, as illustrated in 
Fig. Ela), can be converted to a probabilistic combination of 
many MSCs as in Fig. HJc), the latter of which is equivalent 
to the original MI-SO channel from a detection point of view. 

Proof: Without loss of generality, we assume Y, the 
set of possible received values, is discrete. Since the original 
channel Z^, i-^ Y is symmetric, by Definition]!} there exists 
a bijective transformation T : Y i-^ Y such that T"'-{y) ~ y 
and Vx e Z,n,F{dy\0) = F{T''{dy)\x). Using T, Y can be 
partitioned into many equivalence classes Yj C Y,j G N, 
such that two elements yi and y-i belong to the same class 
Yj if and only if there exists an x E Z„i such that yi = 
T^{y2)- The original MI-SO channel can then be converted 
to an equivalent x ^-^ {j, y) channel as in Fig. |8jb) such 
that y e Yj and j € N. Comparing Figs. |8jb) and (c), 
it remains to show that dP{j\x) does not vary for different 
values of x, and for any j, there exists a MSC(p) with 
parameter p such that F{dy\x,j) and the MSC(p) have the 
same distributions of the a posteriori probabilities regardless 
what type of the a priori distribution P{X) is considered. 
Since the a posteriori probabilities are the sufficient statistics 
of any detection problem, the latter statement implies that 
Figs. |8jb) and (c) are equivalent channels from the detection 
point of view. 

We first show that dP{j\x) is not a function of x. By the 
construction of Yj and by Definition]!} we have 

Vx-eZ™, dP{j\x) = ^ = F(Y,» 

= FiT-^{Y,)\0) 

F{Y,\0) = dPm, 

where (a) follows from the fact that Yj is an equivalence class 
derived from the bijective transformation T. 

The second statement says that for every value j of the chan- 
nel output in Fig.lSjb), there exists a side information value p 
in Fig. Isle) such that the posterior distribution P(a;| Yj , j) from 
Fig. IHb) is identical to the posterior distribution P(2;|y, p) 
from Fig. EJc). To this end, we first let yo denote a "fixed" 
representative element of the non-empty" class Yj from 
Fig.lSjb). We then define yi = T'(yo)- For Fig.|8jc), consider 
a MSC(p) with its parameter vector p ~ {po,--- ,p„i_i) 

"without loss of generality, we may assume Yj is non-empty, since an 
empty class is of little interest. 



defined as follows: 

K cx P{Y = y,\X = 0,y e Y,),Vi e Z,„, (36) 

where "cx" means pi is proportional to the right-hand side 
while satisfying X^iGZ Pi = 1- prior distribution of 
X is the same for all models in Fig. |8] and is denoted by 
P{X = i) = XijVi G Z,„. In the following proofs, it should 
be clear from the context which channel model in Fig. |8] we 
are considering. 

Suppose yi„ E Yj is received for Fig. Ob). Then the a pos- 
teriori probabilities of X = 0, 1, • • • , m— 1 given Yj and j are 
proportional to (P(yjo|0>.?)xo, P(y«o|l>.7)xi, ' ' ' ,P(?/»ol™ - 
l,j)x,„_i). Again by the symmetry of the original channel, 
the a posteriori probabilities can be rewritten as 

(P(y.o|o,j)xo,P(T-i(2/,J|o,j)xi,--- , 

p(T-("-i)(2;,J|X = 0,j)x™-i) 

= (P(y^olo,j)xo,P(y.o-i|o,j)xi,-- - , 

P{yto-{7n-l)\X = 0,j)Xni-l) 
« {PioXQ,Pio~lXl,Pto-2X2, ■ ■ ■ ,Pio-{rn-l)Xvi-l)- 

By noting that the last equation also specifies the a posteriori 
probabilities given Y ^ from a MSC(p) with p specified in 
(in, it is proven that with yi„ and io being the received values 
respectively, the partitioned channel Z,„ i-^ Yj in Fig.|8jb) has 
the same a posteriori probabilities as the MSC(p) in Fig.lSjc). 
To complete the proof that the partitioned channel F{dy\x,j) 
has the same distribution of the a posteriori probabilities as 
the MSC(p), we need only to prove that the probability that 
yi„ G Yj is received (in the partitioned channel F{dy\x,j)) 
is the same as the probability that iq G Z^ is received (in the 
MSC(p)). 

First consider the case in which yi ^ yio^i 7^ io, and we 
then have |Yj| = m. Therefore 

K = P(y = = 0,y G Y,),Vi G Z™, 

instead of being only proportional to the right-hand side. We 
then have 

P{Y = y,^YeY,) 
= J2 P{Y^y^„\X = ^,YeY,)x^ 

= P{Y = y^,-^\X = 0,YeY,)x^ 

~ ^ ^ Pio-iXi 

= P(the output of the MSC(p) is iq). 

For the case in which 3i 7^ iq such that yi = yi^, 
unfortunately, P{Y ~ yia\Y G Yj) does not equal 
P(the output of the MSC(p) is io). However, it can be shown 
that all such i's with yi = will result in the same a 
posteriori probabilities. Furthermore, one can prove that 

p(r = y.jrG Y,) 

= P(the output i of the MSC(p) satisfies yi = yi„). 
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From the above discussion, the distribution of the a poste- 
riori probabilities are the same for the partitioned channel 
F{dy\x,j) and the MSC(p). The proof is thus complete. 



Appendix II 

Proof of the Relationship among CB, SB, and pe 

Without loss of generality, we assume the conditional proba- 
bility P(y |X) is discrete, and all our derivations can be easily 
generalized to continuous/mixed situations. 

Proof of LemmaU] We use q^^y := P{X = x,Y ^ y) 
to denote the joint probability of X = x and Y = y. By 
definition, we have 

CB = 2 y/qa,yqi,y 
y6Y 



SB = 2J2 



yeY 



'2qo,yqi,y 

qo,y + qi,y 



Since for any x,y > 0, min(x, y) < ^ . / < ^/xy, we 

immediately have 2pe < SB < CB. By Jensen's inequality 
and the concavity of the square root function, we can rewrite 
CB as 



CB 



yeY 



■qi,y) 



{qo,y + qi,yY 



< JY.i^o^y + q..)-^^^^ 



' y&Y 



iqo,y + qi,yY 



(37) 



Again by Jensen's inequality and the concavity of the polyno- 
mial f{x) = 4a;(l — x), we have 



qo,y 



qi,y 



qo,y + qi,y qo,y + 91, y 



qo,y + qi.y 



1 - 



min(go.^,gi,.^) 
qo,y + q^y 



SB 

= 'Y(qo,y + qi,y)4: 

yeY 

= +9i,a) 

= 4pe(l-Pe). (38) 

By 03, OH*, and 2pe < SB < CB, the proof of Lemma \7\ 
is complete. ■ 
Proof of Lemma |2} Define Pe,at~*x as the bit error 
probability of the MAP detector given that the input X is 
uniformly distributed on {0, x}, namely. 



(39) 



where Px(e„a(') denotes the probability assuming X is evenly 
distributed on A C Z„j. We note that X {0, x} is equiv- 
alent to a binary-input channel with input alphabet {0,a;}. 



Define CB{0 ^ x) = ^CB{Q 



\CB{x 0) as the 



CB value of the binary channel {0, x} ^ Y. Since CB{Q 



x) ^ CB{x 0), we have CB{0 ^ x) = CB{0 x). By 
Lemma we have 

2pe,0<-»2; < CB{Q X) < 2^ Pefl^xi'^ - Pefi^x)- 

From the above inequalities, the proof of Lemma |2] can be 
completed by proving 

max {pe.Owx} <Pe< Pefi^x- (40) 

x£Z^\{a} 

We need only to prove the result for the MSG case, and the 
proof for general Ml-SO channels then follows by taking the 
probabilistic average of the constituent MSCs. For an MSG 
with the parameter vector p = {po, ■ • ■ iPm-i}, we have 



m — 1 



Pefi^x = 



yTPy+x) 



(41) 



y=o 



Pe = I - max(po,?'l, • • • ,Prn-l) 



Without loss of generality, we may assume pq is the maximum 
entry in p and Pe — 1— Po- Then for any a; 7^ 0, we can rewrite 

Pefl^x as 



Pefi^x = \ P^ + ™n(pj^,p.y+:r)j < 9 ^^Pi: + ^ Pv j 



^ m— 1 



and the first half of Ineq. ( I40t is proved. Also by Eq. ( 14 H and 
the assumption that po is the maximal component of p, we 
have 

Px + P-x 



PeS)^ 



> 



-,VxgZ„A{0}. 



Summing over all possible x e Zm\{0}, the second half of 
(I40> is also proved. ■ 

Appendix III 

Necessary Stability Condition for Zj„ LDPC Codes 

An I'-erasure MSC can be defined by specifying its pa- 
rameter vector p as Po = 1/2, p^ = 1/2 and p^ = 0, 
Vz ^ {0, a;}. Consider an x-erasure Zm "^m MSC, and 
suppose G Zm is received. From these specified conditional 
probabilities, po, • • • ,pm-i, it is impossible for the receiver 
to determine whether the transmitting signal is X = or 
X = —X when a receiving value is F = 0, which is as if we 
are facing a BEC, for which all information regarding X ~ 
and X = —X is erased. However, there is a fundamental 
difference between an x-erasure MSC and a BEC such that 
if we use X — {0, ~x'} for tranmission instead, the x- 
erasure MSC becomes a noiseless perfect channel assuming 
x' 7^ X. In this section, we will use e^; to denote this particular 
parameter p. An x-erasure decomposition lemma is given as 
follows. 

Lemma 4 (x-erasure Decomposition): Consider any MI- 
SO channel with pairwise MAP error Pe.o^x defined in ( I39l l. 
This MI-SO channel can be written as a degraded channel 
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MI-SO Channel 




y ^ Y 



MI-SO Channel 



L J 



(a) By Conditional Distributions (b) With Partitioned Output 

Fig. 8. Three equivalent representations of a MI-SO cliannel. 



MI-SO Channel 



Y Y .- X £ 



dP{p) 



MSC(p) 



JJI^Z,, 



(c) By Prob. Combinations 



of a probabilistic composition of two MSCs, of which 
probabilistic weight dQ'^{q) is defined as follows; 



the 



ri-2pe,o^x if q= (1,0,--- ,0) 
dQ^iq) = < 2pe,0wx if q = 

[ otherwise 
Proof: We need only to prove Lemma 0for an MSC with 
arbitrary parameter p. By taking the average over dP{p), the 
same result holds for general MI-SO channels. 

We first note that dQ'^{-) can be viewed as a Z„j ^ 
({0, 1} X Im) channel, where the first output component is 1 
iff q = Ca;. Let p denote the parameter of the original MSC. 
We would like to show that there exists another channel F 
such that after concatenating dQ'^{ ) and F, we can reproduce 
the probability law p of the original MSC. To be more 
explicit, is a ({0, 1} x Z,„) Z,„ channel such that the 
concatenation Z™ ^ ({0, 1} x Z,„) ^ Z„i becomes an MSC 
with parameter p. We prove the existence of F by explicitly 
specifying its probability law. 

When the first component of the input of F is given, say 
or 1, let the remaining Z,„ ^ Z„j channel be an MSC 
with parameter r or with parameter s (depending on the first 
component being or 1). Define, Vi e Z,„, 



Pi 



; min(p,,pt+a;) - ^min(p,,pt_3,) 



Si = 



2 ■ 

mm{pz,Pz+x) 



It is easy to check that both r and s are valid parameter vectors. 

It is also straightforward to check that the end-to-end 
dQ*^ o F channel is an MSC. By noting that Ip^ Q^x — 
Szez niinlPzjPz+a:)' we can verify that the end-to-end 
channel has the same parameter p as the original MSC. 

■ 

Another necessary lemma is stated as follows. 

Lemma 5 (Monotonicity of p''^\^^}: Let p^'q^^ denote the 
pairwise error probability of the support tree channel of depth 
21 (after / iterations). Then p^'o^^, is non-increasing as a 



function of I. Furthermore, if p 



(0 

e,0- 



> 0, then p^'o^' 



> 0. 



Proof: As I grows, the support tree gives more infor- 
mation by providing additional observations. As a result, the 
MAP error Pe'o^.T non-increasing as a function of I. 

For the second statement, we break one iteration into its 
check node part and its variable node part. Since a check 
node channel is a degraded channel with respect to each of its 
constituent channels, we have (J^^ > p^^\^x > 0, where 



(1+-) 

Pe o*^x is '^he pairwise error probability of the support tree of 
depth 21 + 1 (after incorporating the check node). For variable 
nodes, by the equation CByar = CBin,i • CBi„.2, we have 



Pe,oJx = iff either p^^^^ = or p^ ^, 
> and pl!+ii 



both p^X 



— Pe.O*^ 



= 0. Since 
> 0, it follows that 



0^'+^) >0 ■ 

Proof of Theorem^ Suppose Theorem]3\is false, namely, 
there exists a MI-SO channel such that limj^oo ' = while 
there exists an xq G Z„i\{0} satisfying X2p'{l)CB{Q 
xo) > 1- By Lemmata^^ and0 we have limj^oo Pe ow^o ~ 
and Pe^o^xo > 0, V/ G N, namely, {Pe.o^jj^lzeN is a strictly 
positive sequence with Umit 0. Therefore, for a sufficiently 



Co) 
e,0<- 



< e. Without 



= e > 



small e > 0, there exists an Iq such that 
loss of generality, we may assume 

By Lemma we can replace a supporting tree channel of 
depth 2/0 by a probabilistic combination of a perfect channel 
and an xo-erasure channel with weights (1 — 2e, 2e), which is 
denoted by dQ'^{q). Similarly, for a supporting tree channel of 
depth 2(^0 + Al), we can replace each of its youngest subtrees 
of depth 2lo by a dQ'^{q) channel, so that after subsititution, 
the new channel becomes a supporting tree channel with depth 
2 A/ and all its youngest descendants are dQ'^{q) channels. We 
then use Pe'o^'t? 'Ze'owajo denote the pairwise error 
probabilities of the original tree channel of depth 2(/o + A/) 
and the new channel of depth 2AI respectively. By the channel 



degradation argument, we have p^ g^'^'"' 



(Ai) 
?e,0<-+a;o ' 

It is worth noting that p'^^o'^^o = 9e°dwxo 

For notational simplicity, we define the output of an xq- 
erasure channel to be +0 if the output satisfies Y ^ X + 0. 
Similarly, an xo-erasure channel outputs +a;o if the output 
satisfies Y ^ X + xq. For Al = 1, we consider the new 
support tree channel of depth 2, namely, only one iteration 
of check node and variable node is considered. Readers are 
referred to Fig.@]for illustration of a regular (2,3) LDPC code, 
in which both dP^ and dP^ should have the form of dQ'^ {q). 
Each check node constituent channel (CNCC), that is dP^ or 
dPg in Fig. |4] can be either a noiseless perfect channel (with 
probability 1 — 2e) or an .XQ-erasure (with probability 2e). If 
none of the check node constituent channels (CNCCs) is .tq- 
erasure, we can successfully decode the original input X with 
no error. The cases that more than two CNCCs are .tq -erasure 
only happen with probability 0{e^), and are of less importance 
in the asymptotic analysis. Therefore we focus only on the case 
in which one and only one CNCC is xo -erasure. Furthermore, 
since the input X and the input of the individiual CNCC satisfy 
a parity check equation, the only sub-case in which there is 
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no additional information (from the CNCCs) distinguishing 
X = xq from X = is when the -erasure CNCC has an 
output providing no information for detecting —xq from 0. We 
then have ( I42> . 

The inequality (a) in (142 > follows from the fact that +0 
is an output that the xo-erasure CNCC cannot distinguish 
—xo from 0. Therefore, the event of misdetecting X = 
by X = xo contains the case in which the CNCC outputs 
+0 given X — 0. Similarly, +xq is an output providing no 
information distinguishing from xq, which is thus contained 
by the event of misdetecting X = xq hy X = 0. 2eX2p'{l) 
is the probability that one and only one CNCC is xo-erasure. 



26A2p'(l) 



corresponds to the probability for which the xq- 

- l^n- P{Y=y\X=0) 



P{Y=y\X. 



and 

erasure channel outputs +0 (or +a;o). = log 
is the LLR between X = and X ~ x, and dPx{-) is the 
density of the initial LLR message rrix given X = 0. Equality 
(b) in \A2\ follows from the fact that with the only .xq -erasure 
CNCC providing no information, misdectection happens when 
the original channel observation also provides an incorrect 
LLR message. 

Note: If Xq 7^ ~xq in Z^, the inequality becomes an 
equality. If xq ~ — in Z„j, then (Zg^o^^.^ equals twice the 
right-hand side of the above expression. 

By similar arguments, the second iteration gives 

^ \J 7n—~oo 



m— — oo 



and after 2AI iterations we have 



2AI 



It is easy to show that {dP^xo ® dP^g) is a symmetric 
distribution defined in [6], i.e., dP{m) = e"^dP{—in), and 
its Bhattacharyya noise parameter is {CB{0 — > Xq))'^. Choose 
S > such that A2/9'(1)(CP(0 xa) - S) > 1. By the 
tightness of the Bhattacharyya noise parameter, we can lower 



bound ql^^xa sufficiently large Al by 



> 



e(A2p'(l))'^' (CBiO 



Choose sufficiently 



Al 



2Al 



such that 
2 and sufficiently 



large 

(A2p'(l)(Ci?(0^xo2-'5))'^' > 

small e, we have g^^otllo > 2e + ©(e^) > e. By the channel 
degradation argument discussed earlier, we have 



(io+2A0 



(2A0 



> e 



which contradicts the monotonicity result in Lemma Using 
this contradiction, the proof of Tlieorem^is complete. 

■ 

Appendix IV 
The Maximizing Distribution for Check Nodes 
WITH Constraints on {CB^mSBin) 
Proof: We take the approach of considering the marginal 
dP first and assuming that dQ is a point mass, i.e., dQ 



concentrates all its probability on a fixed q^. To simplify the 
notation, we let a := 2yjp{\ — p) and b := 2-^go(l — Qo)^ and 
drop the subscript 1 in {CBin.i, SBin.i) to {CBin, SBin). 
The original problem (13 1> becomes a linear programming 
problem on dP: 



max C 



v/a2(l- 62) + 62jp(a) 



subject to / dP{a) 

la=0 



[ adP{a) < CB^n 

Ja=0 

I a^dP{a) < SB^n 
Ja=a 

dP{a) > 0, Va e [0,1]. 

Note: From the BSC decomposition perspective, dP{-) denotes 
the probabilistic weight for different BSCs, which can be 
indexed by p or by a = 2^p{l — p) at one's will. Previously, 
dP{-) denotes the probabilistic weight for different BSCs 
indexed by p. Here the notation is slightly abused so that 
dP{ ) also denotes the probabilistic weight for different BSCs 
indexed by a = 2\Jp{l — p). 

The corresponding dual problem is 



min ^ := yo + ViCBin + y2SBr, 
yo.v1.y2 



subject to ya + ayi + a^y2 > \^a^{l - b'^) + b^, Va € [0, 1] 
J/1,2/2 > 0. 



Let 



t = 

2/0 = 

2/1 = 

2/2 = 



SBin 

C Bin 

2 ( f2(i_52)^2&2 



t I 2^*2(1-62) +62 



-b 



^t^{l-b^) + b2 



It is easy to check that both 2/1,2/2 ^ 0- iemmfl|6| (stated at 
the end of this proof), y* = (j/q ,2/1,2/2) is a feasible solution. 
Let 



dP*{a) 



t 







if a = t 
otherwise 



It can be verified that the duality gap between the two feasible 
solutions y* and dP* is zero. By the weak duality theorem of 
linear programming, dP* is the maximizing distribution when 
dQ concentrates on go- Since dP* does not depend on 6 (and 
thus does not depend on q^), dP* is the universal maximizer 
for general dQ. ■ 
Lemma 6: j/q + ayl + a'^y'2 > ^ {1 — b'^) + b'^ for all 
a,be [0, 1]. 

Proof: Let /(a) 2/o +"2/1* +"^2/2 - ^^2(1 - 62) + 52 
be a function of a while b and t are fixed parameters. We first 
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le'Lxo = PiX ^0)Pi^XMAP = Xo\X ^Oj +P{X = xo)P[Xmap^O\X = xo) (42) 

(a) 1 

> -P(one and only one CNCC is xo-erasure and that channel outputs +0|X = 0) 



• P I^Xmap ~ xo\one and only one CNCC is xo-erasure and outputs +0, X = 
+ -P(one and only one CNCC is XQ-erasure and outputs +.to|X = xq) 

■ P (^Xmap — O|one and only one CNCC is xo-erasure and outputs +a;o, X — xq 

(6) 1 /2eA2p'(l)\ , ,1 /2eA2p'(l)^ 



rn^n — — oo \ / ^ '>n-x,-, — — oo 



note that 



3a62(l - fo' 



2\2 



Va2(l-62) + , 



>0, (43) 



/(O) 



and 



da 



0, 
0, 

0. 



By simple calculus, the conclusion /(a) > 0,Va e [0,1] can 
be obtained in different ways, one of which is demonstrated 
as follows. 

We first show by contradiction that there exists no «□ other 
than and t such that /(ao) = 0. Suppose there exists an 
flo G (0,1), such that oq 7^ 0,t, and /(ap) = 0. Since 
/(O) = fit) = 0, by the mean value theorem (MVT), 
there exist ai < a2 e (0, max(ao, t)) such that ai,a2 ^ t, 
and /'(ai) = /'(oa) = 0. Since f'{i) = 0, by the MVT, 
there exist as < 04 G (min(ai, <), niax(a2, i)) such that 
/('Has) = f^^\ai) = 0. Again by the MVT, Bag G (a3,a4), 
such that f'^^\a^) = 0. By j43> , the only possibility that such 
an 05 G (0, 1) exists is when /'^■'(•) = is a zero-function, 
which contradicts the assumption that the minimal number of 
distinct roots is no less than 3 (with values 0, t, and ao). 

Since there exists no ag other than and t such that 
/(ao) = 0, the only case that 3a S [0, 1], /(a) < is when 
one of the following statements holds: (i) /(a) < for all 
a e (0, t), or (ii) /(a) < for all a G (<, 1]. Suppose (i) holds. 
A contradiction can be obtained by consecutively applying the 
MVT as follows. Since /(O) = f{t) = 0, 3ai G (0,t) such 
that /'(ai) = fit) = 0. Therefore, 3a2 G (ai,<) such that 
/"(a2) = and /(a2) < 0. Since f{t) = 0, Bag G (a2,t) 
such that /'(as) > 0. Since f'{t) = 0, 3a4 G (as,*) such that 
/"(a4) < 0. Therefore, 3a5 G (a2, 04) such that f'^^\a5) < 0, 
which contradicts j43> . 

The remaining case is when (ii) holds and /(a) > 0,Va G 
(0,i). Since f{t) = 0, by the MVT, 3ai G (0,0, a2 G (t, 1] 
such that /'(ai) < and /'(as) < 0. Since f {t) = 0, 
3a3 G (ai,t),a4 G (<, 02) such that f"[a^) > and 
/' (a4) < 0. Therefore 3a5 G (03, a4) such that /(^^(as) < 0, 
which contradicts ( I43l l. From the above discussion, the proof 
is complete. ■ 



Appendix V 

The Upper Bounding Distribution for Variable 
Nodes with Constraints on {CB,n, SBin) 

We take the approach of assuming dQ concentrates on a 
fixed go- Let a 2^-pi\ — p) and h 2-\/go(l — go) 
and drop the subscript 1 in {CBin.i, SBin.i) to write 
{CBin, SBin). The original problem i33i becomes a linear 
programming problem with the primal and dual representa- 
tions as follows. 



max C 



a2(l-62) + &2 
subject to / dP(a) = 1 



dP{a) 



J adP{a) < CB,„ 

J a^dP{a) < SB,„ 
dP{a) > 0, Va G [0, 1] 



min ^ yo + yiC Bin ^ 

Vo,yi,y2 

subject to yo + o-Vi + a^y2 > 



yiSBi, 
a" 



a2(l-62)+62 



, Va G [0, 1] 



2/1, y2 > 0. 

For convenience, we define t := SBin /CBin and rf,(a) := 

a^(l-h^)+b^ ■ 

Unlike the check node channel case, this time the op- 
timal primal solution dP*{a) depends on the value of h, 
and different values of h will lead to different closed form 
solutions of the optimizer dP*{a). The effect of different 
&'s can be summarized as three different cas_es i n which h 
belongs to one of the following three intervals 



CBl 



CBf 
1+CB'i 



and 



Proposition 2: If 6 G 



0, 



CBf 
l + CB't 



0' \l l + CB'i 

, respectively. 

the maximizing 
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dP* (a) and the optimum values are as follows. 

1 if a = CBin 
otherwise ' 



dP*ia) 



(44) 



Proof: It is easy to check that the specified dP*{a) is 
feasible. By choosing 



CBl{l~b^)+b-^- 



Proof: It is easy to check that the specified dP{a) is 



62 



■a is the 



feasible. We then note that r^, (0) = 0, and 
only tangent line of rb{a) passing through the origin (with the 
contact point at = \J i^y^ )- Furthermore, when a > ^ 
we have 



2/0 
Vi 

y*2 







> 
> 0, 



26'*7^77T^^rl^— TTTT < and rt(a) is thus a 



(t2(l-62)+^2) 

we have ^* = ^2(f!^t)2)^b2 = C*- So it remains to show that 
y* = {VoiVitV2) is feasible for all b e [0,1]. Let g{a) 



concave function in the interval 
observations. 



[J . 2 , 1]. From the above 




Ub{a) = 

is the convex hull of ri,{a). By Jensen's inequality, 

rb{a)dP*{a)< [ Ub{a)dP*{a) 



(45) 



d^ 

da^ 
5(0) 



rij{a). By the following observations 

24a64(l - 52) 



(a2(l-62) + 62) 
> if a e [0 



^{b'-a^l-b')) 



dg_ 

da 






0, 



J adP*{a] 



CBl{l-U^) + b^- 
(46) 

Since dP* (a) in (I44> achieves the upper bound in ( I46> . it is 
indeed the maximizing dist ribution 

CB't 



we have .g(a) > for all b G [0, 1], a e [0, 



fe2 



], which 



can be proved by exactly the same argument a s in th e proof of 



Lemma |6| We then consider the case a G [ 



fc2 



, 1]. Using 



Proposition 3: If 6 G 



dP* [a) and the optimum values are as follows 



dP*{a) = 



a similar argum ent ba sed on the MVT as in Lemma |6| we 
> by first showing g"{a) > 0,Va e 



, the maximizing ^^^^ P^'^^^ 9 — 



4 3a^(l-b^)-b^ 



> 




(47) a e [ 



[t, \l T^]- By noting that g"{a) = 2|/^ +2fr- ^^2(i_j,2)+b2)d 
for all a e [\f^^, 1]^ we conclude that g'{a) > for all 
1]. Since g I 



> has been proved in the 



b^ 



, ll. From 



r = 



Proof: It is easy to check that the specified dP*{a) 
is feasible. By again invoking Jensen's inequality on Ub{a) 
defined in ( 145 > . we have 



first case, we then have g{a) > for all a S [ 
the above reasoning, we have g{a) > for all a,b G [0, 1], 
and thus y* is feasible and the proposition follows. ■ 
From Propositions^to^ we have the following tight upper 
bound: 



rb{a)dP*{a) = 



I'b^ 



rb{a)dP*{a) < / Ub{a)dP* {a) 
< Ub [ I adP*{a) 



a2(l-62) + 52 

s{CBin,SBin, b), 



dP*{a) 



CB,, 



&2 



where 



1-62' 



(48) 

Since dP* (a) in (I47> achieves the upper bound in ( I48t . it is 
indeed the maximizing distribution. ■ 

, the maximizing dP* (a) 



s(CB,n,SB,nM 

CBf^{l-b^)+b^ 
CBi, 



1 



Proposition 4: If 6 S 
and the optimum values are as follows 

CB 



2 V l^**' 
SBj^b'^ 



if 6e 
if e 
if 6e 



0, 



/ CBf^ 




V 1+CBf^ 




CB'f^ 
l+CB'-' 


'V 




1 







(49) 



SB,, 



dP*{a) 

c 



CB 



SB,„ 



if a = t 
if a = 
otherwise 

sB„y 



t2 (1 - &2) + 52 



Hereafter, we will show that the 6- value-independent dP**{a) 
in i34\ is an upper bounding distribution, such that dP**{a) 
may not be feasible, but the resulting J rb{a)dP** {a) is no 
smaller than s{CBm, SBin,b) for all b G [0, 1]. 

Lemma 7: J rb{a)dP**{a) > s{CB^„ , SBi^.b) for all b e 
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Proof: By the monotonicity of r;, (a) as a function of a, 
we have 



n{a)dP**{a) 



t 



-n 



iCB,„) + fsBniVSB~) 



t + CBm 

> (1 - fsB)—^^n{CB,n) + fsBrb{CB,n) 

= rfa(CS,„) = s(CS„,5B,„,6). 



Lemma 8: J ri,{a)dP** {a) > s{CBin, SBin,b) for all e 



1+CBi 



Proof: We prove this b y dire ctly applying calculus. By 
changing variables to x J -j-^ and using c as a shortcut of 



CBin (note that SBin = tc), proving Lemma |S| is equivalent 
to showing 



rb{a)dP**{a) 
(l-fsB) * 



+ (1 - /sb) 



t + c + x'^ 
t^x^ 



+ fsB 



tcx 



tc + x^ 



c 



tcx 



X 



t + ct^ + x^ 

4 + t2^2 + ^2 (^2 + c2 - (1 - " c)^) ' 



(x2+t2)(-2.2+ic)(a;2 + c2) 



>s(CB,„,5S,„,5) 

= |x, Vxe [c,t] c [0,1]. 

Multiplying the common denominator and changing the vari- 
able to y := the desired inequality becomes 

t^c^y^ + t^c^{t^ + + tc)y^ + t^c^{t^ + + tc)y'^ + t^c^ 
- 2tVTcy{t\Hy^ + 1) + tcy''{e + - (1 - - c)^)) 

<0, 

for all ^ < y < again changing the variable to 

w := Vtc{y + we would Uke to prove that 

VM - '2fsBt{t -cf<0, Vw e [2 Vte, {t + c)] , 
where ri{w) is defined in (I35> . By noting that 77(i + c) — 



-2(t - c){c{t + c) + ft{t - c)) < for all 



fsB S [0, 1], we would like to show that there exists no root of 
r]{w)-2fsBt{t~cf in [2y/U:.,t+c]. \ft-^c{2t-c) < 2^/te, 
then by definition /ss = 0. By simple calculus, there is no 
root in [2^/tc, t + c]. If 2\/tc -t + y/c{2t - c) < 0, there is 
one root of rjiw) in [2^/tc,t + c]. By letting fsB ^ ''^^ ^ 
where 



2t{t-c)'^ 



W 



2t-^4t2-3(t-c)2 
. 3 



if r]'{2Vtc) < 
otherwise 



we guarantee that 7]{w) — 2fgBt{t - 
i]{w), has no root in [2^/tc, t + c\. 



c)^, the shifted version of 
This completes the proof. 



Lemma 9: 
11. 



/ n{a)dP**{a) > s{CB,„,SB,„,b) for all b e 



t2 



Proof: In this proof, we use another index a = = 
4p(l — p) for different BSCs and dP{a) now denotes the 
corresponding probabilistic weight for BSCs indexed by a, 
which is different from the dP{p) and dP{a) discussed 
previously. We can rewrite rb{a) with respect to the new 
index a such that it becomes ri,{a) — -^^^fj-r^ryf^ ^ the SB 
value when the constituent BSCs are indexed by a and b 
respectively. 

d^r (a) 

It can be shown that < and ri,{a) is a con- 

cave function of a. By noting that J adP**{a) = SBin 
and the weights in dP**{Q) are concentrated only on three 
points a = CBf^, SBi„, and t^ in an increasing order, we 
have J ri,{a)dP** (a) > Ai, where Ai is the intersection 
of the vertical line a = SBi„ and the chord connecting 
{CBl.niCBl)) and {t^,n{t^)). 

We also notice that s{CBin, SB^, b) is the intersection of 
the vertical line a = SBin and the chord connecting (0, fb(0)) 
and (t^, rf,(i^)). By the concavity of rb{a), we conclude 
/ n{a)dP**ia) >Ai> s{CB,n,SB,n,b). ■ 

Appendix VI 
Proof of Theorem |S| 

We provide a proof of a more general theorem, which 
includes general error correcting codes and multiuser detection 
as special cases and is formally stated as follows. 

As in Fig. |9] consider any deterministic/randomized se- 
quence mapper'^ C : {0,1} ^ {0,1}" and W = C{X). 
Each coordinate Wi of W is passed through independent BI- 
SO channels Fi := {fi(ii\w)} to generate the observation Yi, 
i e {1, 2, • • • , n}. Let X{Y) be the MAP detector, and define 



p,{{F,}) := PxMX{Y)^X) 
and CB{{F,}) Ex,y 



'P{X\Y) 



P{X\Y) J 

as the error probability and CB value of this X 1-^ Y vector 
channel given the conditional channel distributions {Fi}. We 
then have the following theorem. 

Theorem 9: For any uniforni/nonuniform binary input dis- 
tribution on X, we have 

CB{{Fi})<CB{{FBsc,p.}), 

where for any i, pi satisfies 4pi(l —pi) = / 4p(l — p)dPi{p). 
The integrator dPi{p) is the equivalent probabilistic weight 
in the BSC decomposition of channel Fi as described in 
Section III-A.2I 

Theorem^is, a special case of r/ieoremlP] obtained by letting 
X ^ C{X) be the binary-input/vector-output support tree 
channel. 

'^X and C{X) can be regarded as a binary-input vector-output cliannel. 
Or C{X) is the subspace of codewords corresponding to information bit X. 



DRAFT FOR IEEE TRANS. INFORM. THEORY 



23 



X 



c 



Ml 



fi{y\w) 
f2{y\w) 



fn{y\w) 



X{Y) 



X 



Fig. 9. General deterministic/randomized bit to sequence mapper witli 
independent symmetric channels. 



Note 1: In the setting of Theorem |P] we only require all 
constituent channels to be of BI-SO type. The bit-to-sequence 
mapper X i-^ C{X) does not need to be symmetric, which is 
different from the case of LDPC codes. 

Note 2: The definition of SB in (|3 is valid for general 
BI-NSO channels with arbitrary input distributions. However, 
with a non-uniform input distribution, SB{Fbsc,p) 7^ 4p(l — 
p). This is the reason why in Theorem^ we deliberately use 
4ft(l -pO = / Ap{l - p)dP,{p) instead of SB{Fbsc,p.) = 
SBiF,). 

Proof of Theorem |9} By rewriting each BI-SO channel 
Fi as the probabilistic combination of BSCs with weights 
dPi{pi), each observation yi can be viewed as a pair {zi,pi) £ 
{0, 1} X [0, 1/2], where Zi is the binary output of Fssc.pi ^i^^ 
Pi is the side information specifying the crossover probability 
of the corresponding BSC. Taking the marginal approach, we 
will focus on 2/1 = {zi,pi) and treat all y2, ^3, 2/4, • ' ' , Vn as 
the side information s. The conditional probability P( |S — 
s,pi) can then be factored as 

P(X,W^i,Zi|S = s,pi) 

= {pi + (1 - 2pi)6{Z, - Wi)) P{X,Wi\S = s,pi) 

= (pi + {1 - 2pi)S{Z, - Wi))P{X,Wi\S = s), (50) 

where 6{x) = 1 iff a; = 0. To write P{X,Wi\^,Pi) = 
P{X, W^i |S), we use the fact that knowing what type of BSCs 
we are facing (namely, knowing p) provides no information'^^ 
about the input distribution (Wi). This fact also implies that 
dPi [p] does not depend on the distribution of S either. As a 
result, we have 

P(S,pi) - P(S)P(pi) = P(S)dPi(pi). (51) 

By j50> and ( 15 1> . the corresponding factor graph is drawn 
in Fig. [TO] We can rewrite the conditional distribution 
P{X, Wi\S) in the matrix form: 

P(VKi = 0,X 0|S s) P(W^i = 0,X = 1|S = s) 
P(VKi = 1,X = 0|S = s) P(W^i 1,X = ijs = s) 

a b 
c d 

where a, b, c, and d are functions of s satisfying a,b,c,d> 
and a + b + c + d = 1. It is worth repeating that a, b, c, and d 

^^dP(p) only depends on the channel distribution f{z\w), not on the a 
priori distribution of W . Thi s is a special property of the BSC decomposition 
mentioned in Section l!l-A.2l For BI-NSO channels, though the corresponding 
BNSC decomposition can be found as in Section lV-AI the probabilistic weight 
dP(po^iiPi^o) depends on the distribution of W. 



X 



P(S) 



P{X,Wi\^) 



dPiipi 



Pi 



FbSCpi 



Fig. 10. The factor graph of the five random variables: X, Z\, W\, S, and 
Pi- 



do not depend on p\ . The conditional input-output distribution 

P(X, Zi|S,pi) then becomes 

P(Zi = 0,X = 0|S = s,pi) P(Zi = 0,X = 1|S = s,pi) 
P(Zi = \,X = 0|S = s,pi) P(Zi = 1, X = 1|S = s,pi) 

a(l-pi) + cpi 6(l-pi) + cipi 
api+c(l-pi) 6pi + a!(l-pi) 

The value of CB for the X Y channel (or equivalently 
X ^ (Zi,pi,S)) becomes CB = E | |. 
Taking the expectation step by step, we have 

CBx,Zi\pi,s 



-X,Zi\pi.S 



^ P(X|Zi,pi,S) 
P(X|Zi,pi,S) 



= 2v/(a(l - pi) + cpi) (5(1 - pi) + dpi) 
+ 2v/(aj5i + c(l - pi)) {bpi + d{l - pi)). 

By Proposition (stated at the end of this proof), 
CBx.Zi\pi,s is a concave function of f3 := 4pi(l —pi) for all 
valid a, b, c, and d. By Jensen's inequality, for any channel 

Fc, 



CB 



X,Zi,pi\S 



I / P(X|Zi,pi,S) 

.1/2 

/ CBx.z,\p„sdPi{pi) 

Jp=Q 



< CB 



(52) 



where pi is the crossover probability such that 4pi(l —pi) ~ 
J 4:pi{l-pi)dPi{pi). By (|53 and noting that Fbsc,pi is the 
universal maximizing distribution for any realization of S, we 
obtain that 

CB{{Fi,F2r-- ,Fn})^ J CBx,ZuP,\sdP{S) 
< J CBx.z,\p„sdPi^) 

= CB{{FbSC,pi, F2, F3, • • • , Fn}). 

By repeatedly applying this Ci3-increasing channel replace- 
ment until all constituent channels Fi are replaced by Fsscpi > 
the proof of Theorem |P| is complete. ■ 

Proposition 5: For any constants a,b,c,d > and p G 
[0,1/2], we have 

v/(a(l-p)+cp) {b{l-p)+dp) 
+ Viap + c(l - p)) {bp + d{l - p)) 



DRAFT FOR IEEE TRANS. INFORM. THEORY 



24 



is a concave function of (3 := 4p{l — p). 

Proof: This proof involves several changes of variables. 
It is worth noting that this proposition is a pure algebraic 
statement and the notations involved herein are irrelevant to 
those of the LDPC code problem. 

We first let X = 1 
the problem becomes to prove that both 



^ = and i? = Then 



f{X) = ^JA + X^JB + X + ^JA - X^/B - X 



and 



-i{X) = -sjA - xVb + X + VA + xVb - X 



are concave functions of /? = 1 — X"^, for all A,Bg [1, oo] 
and X e [0, 1]. We focus on the concavity of f{X) first. Using 
the chain rule, 



df{X) _ dfiX) dX 



d/3 



dX df3 



1 1 

2 ' 2X 



B + X 
A + X 



A-X 
B -X 



B-X 
A-X 



A^X 

B + X ' 

^-ff^ = ~JxA^^' showing the concavity of 
f{X) as a function of p is equivalent to showing: 

d df{X) 



Since 



dX dp 
1 



A + S + X + f |±§ + |±f 



4X2 



^iA + X){B + X) 
-A-B + X + ^ ^^-^ ' 



-X 



A-X 



^{A-X){B-X) 



4X' 



:f2{X) > 0. 



To show f2iX) > 0, we first note that /2(0) = 0. Its first 
derivative is 

df2{X) ^ 3{A-B)^X 
dX 4 

f (A-X) + {B-X) {A + X) + {B + X)\ 



\^iA-X)iB-X) ^iA + X)iB + X) J 

By Lemma ^O] (stated at the end of this proof), we have 
> 0. Thus f2{X) > 0, which implies that f{X) is 
concave as a function of /?. 
For g{X), we have 

dgjX) ^ 1 _1_ 
dp 2' 2X 



A + X 
B-X 



^.ince rf^i - 



B-X 



A + X 

1 d dg(X) 
2X dX d/3 



\Ib-, 


X 




4) 











showing the concavity of 



g{X) is equivalent to showing 

d dg{X) 



dX dp 
1 



-A + B- X 



X I A+X , B-X 
2 V B-X ' A+X 



4X2 



^{A + X){B-X) 
A - B - X ' ^ ^ -4-^ ' 



2 V B+X ' A-X 



^{A-X){B + X) 



AX' 



r.92(X) > 0. 



To show g2{X) > 0, we first note that 52(0) = 0. Its first 
derivative is 

dg2{X) _ 3(A + B)2X 
dX ~ 4 

( (A + X) - {B - X) {B + X)-{A-X)\ 



\^iA + X)iB-Xy ^{A-X){B + XY ) 

By Lemma^O\ we have '^^^'^^ > 0, which implies that g{X) 
is concave as a function of p. This completes the proof of 
Proposition ]5\ ■ 
Lemma 10: For all a,b,c> 0, we have 

a + b ^ (a + c) + (6 + c) 



(a6)5/2 - ((a + c)(6 + c))5/2 
{a + c) — b {b + c) — a 



> 0. 



((a + c)6)5/2 ((& + c)a)5/2 

Proof: By noting that^ = i + i>^ + ^ = 

^"a'^+lxl+'c)' ^"^1 < a& < {a + c){b + c), we prove the first 
inequality. 

For the second inequality, without loss of generality, we 
assume a < b. We then observe that 

(a + c)b > (6 + c)a > 
{b + c)-a 
{b + c)a ~ 

(a + c)-b (b + c)-a 1 1 1 1 

h = 1 > 0. 

(a + c)b {b + c)a b a + c a b + c 

(53) 

Considering ( I53t . after multiplying the non-negative second 
term 



{b+c}-a 
ib+c) 



— by a larger factor 



\/{b+c)a 



and the possibly- 



neeative first term by a smaller factor _ ^ ^ , the 

new weighted sum is no less than zero, namely, 

{a + c)-b {b + c)-a ^ 
{{a + c)bf/^ ((6 + c)a)5/2 " " 

This completes the proof. ■ 
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